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ABSTRACT 

Contractor estimating systems that produce reliable proposals are a key 
safeguard for the Government to obtain fair and reasonable contract prices for goods 
and services. Government Administrative Contracting Officers (ACO), with the help 
of the Defense Contract Audit Agency (DCAA), are responsible for determining the 
adequacy of contractor’s estimating systems. If a DCAA audit finds estimating 
systems deficiencies, the ACO may take several actions including disapproving the 
contractor’s system, in part or in whole. The process of formal disapproval and 
deficiencies resolution is a series of contractor written responses, corrective action 
plans and ACO evaluations. This process can take years before a final determination 
is made. This thesis will focus on how the Procuring Contracting Officer (PCO) 
obtains the information about deficiencies or disapproval and what actions he/she 
takes to insure the Government is receining a fair and reasonable price for the goods 
and services it buys. 
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I. INTRODUCTION 



A. BACKGROUND 

When contractors submit proposals in response to a Government Request for 
Proposals (RFP), their proposals must be evaluated to determine which one represents the 
best value to the Government. The evaluation factors on which the proposals are scored can 
vary in number and type. Cost, past performance, and quality must be included as factors. 
When evaluating the cost factor, the Procuring Contracting Officer (PCO), must evaluate the 
contractor’s cost estimates of the product or service. 

Cost estimating systems that produce reliable price proposals are a key safeguard to 
obtaining fair and reasonable contract prices (GAO 95-3 , 1 995 : p. 11). When a contract price 
is higher because it is based on inaccurate, incomplete, and non-current data, the Government 
has the right to a price reduction under the provisions of the Truth in Negotiations Act 
(TINA). When contracts are overpriced because of poor estimating, the Government may 
not recover the overstated price under TINA. 

Government Administrative Contracting Officers (ACOs), with help from auditors 
in the Defense Contract Audit Agency (DCAA), are responsible for determining the 
adequacy of contractors’ estimating systems. When the DCAA reports an estimating 
deficiency, DoD regulations establish procedures and time frames for their correction. If a 
contractor does not make adequate progress in correcting the deficiency, ACOs are 
authorized to take actions to obtain correction, such as reducing or suspending progress 
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payments, recommending nonaward of potential contracts, or disapproving all or part of the 
estimating system (DFARS, 1996: 215-811-70). 

The process of formal disapproval is a series of contractor’s written responses to 
DCAA audit findings, corrective action plans, and ACO evaluations. This process can take 
years before a final determination is made (Rich Buhre interview). The final notice of 
disapproval must be sent to each Contracting Officer and Contract Administrative Office 
having substantial business with the contractor. 

Once notified, PCOs negotiating proposals generated by an estimating system with 
an identified deficiency shall evaluate whether the deficiency impacts the negotiations. If 
it does, the PCO has several alternatives provided in the Defense Federal Acquisition 
Regulation Supplement (DFARS): 

(1) Allowing the contractor additional time to correct the estimating system 

deficiency and submit a corrected proposal. 

(2) Considering another type of contract. 

(3) Using additional cost analysis techniques to determine the reasonableness 

of the cost elements affected by the system’s deficiency. 

(4) Segregating the questionable areas as a cost reimbursable line item. 

(5) Reducing the negotiation objective for profit or fee. 

(6) Including a contract (reopener) clause that provides for adjustment of the 

contract amount after an award. (DFARS, 1996:215.81 1) 

Numerous General Accounting Office (GAO) reports have identified poor contractor 
cost estimating systems as a significant problem in Government contracting. These 
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inadequate systems have potentially cost the Government $3.67 billion between the years 
1987 and 1991 (GAO 95-3, 1995: p. 10). 

B. OBJECTIVES OF THE RESEARCH 

The area of Contractor Cost Estimating Systems is far too large for a single thesis. 
Thus, this thesis will focus on how PCOs are informed of estimating system disapprovals and 
deficiencies and what actions they might take after this information is obtained. The 
objectives of this research effort will be from the Government’s perspective, including: 

1 . How PCOs obtain information of poor contractor cost estimating systems. 

2. How this information is used once obtained. 

C. RESEARCH QUESTION 

In order to accomplish the objectives of this research effort, the following primary 
research question is addressed: 

What action does the Procuring Contracting Officer take when notified that a 
contractor’s estimating system has deficiencies or has been disapproved by the 
Administrative Contracting Officer? 

The following subsidiary questions were applicable to this research effort: 

1. Are the alternative actions specified in DFARS 215.81 l-70(g)(2) adequate to 
motivate a contractor to respond to the PCO’s desires? 

2. Are there more effective actions currently being used by PCOs and if so 
should they be incorporated into the DFARS? 



3 



3. If the PCO gathers independent data indicating a contractor has estimating 
system deficiencies, how does this affect the performance of his or her duties? 

D. SCOPE OF THE RESEARCH 

This study focuses on the collection methods and usage of information on contractor 
cost estimating systems by PCOs. Analysis is limited to these two areas to maintain a 
manageable research topic size. To collect data for analysis, a survey was conducted of 
various procurement programs and projects with Research, Development, Testing and 
Evaluation (RDT&E), budgets of more than $25 million throughout DoD. Additionally, 
interviews have been conducted with DCAA, Defense Contract Management Command 
(DCMC), and GAO personnel. The use of activities nationwide within DoD provides a 
broad statistical base for Government represen tatbn. Contact via correspondence provided 
sufficient input. By using questionnaires to gain data for analysis, the researcher assumed 
that respondees were answering truthfully and in full. 

E. RESEARCH METHODOLOGY 

The researcher used three methods of data collection to complete the this effort. 
First, a comprehensive review of the available literature was conducted utilizing the Naval 
Postgraduate School (Knox) Library, the Systems Management Acquisition Library, the 
Defense Logistics Studies Information Exchange (DLSIE), and the Defense Technical 
Information Center (DTIC). Next, surveys were sent to selected contracting personnel at 
procurement programs and projects with RDT&E budgets of more than $25 million. 
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Surveys were developed through research using previously discussed sources. Lastly, 
telephone interviews were also conducted on a limited basis to clarify and enhance responses 
to the survey. 

F. ORGANIZATION 

The research has been divided into six chapters. In this Chapter, the objectives of the 
research have been set forth, the scope and direction of the effort identified, and the 
methodologies for data collection presented. 

Chapter II discusses the regulatory environment applicable to cost estimating 
systems. 

Chapter III explores cost estimating systems and summarizes a review of the 
literature. 

Chapter IV presents the survey questionnaires sent to the various activities, the 
responses received, and an analysis of the data. 

Chapter V presents the principal findings, conclusions, and recommendations 
generated by this research. 
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II. COST ESTIMATING REGULATORY ENVIRONMENT 



A. INTRODUCTION 

DoD annually awards billions of dollars in contracts for a wide variety of products 
and services. Many of these procurements are conducted in the absence of the competitive 
marketplace that establishes fair and reasonable contract prices. Therefore, DoD Contractii^ 
Officers rely on price proposals developed and submitted by contractors to a great extent. 
Price proposals are developed by contractors from their cost estimating systems. Therefore, 
sound estimating systems are fundamental in determining fair and reasonable contract prices. 

Recognizing the Govemment’svulnerabilityin negotiating noncompetitive contract 
prices, the Congress passed the Truth-in-NegotiationsAct (TINA) in 1 962 (GAO 93-1,1 992: 
p. 1). The Act, intended to protect the Government against inflated cost estimates, required 
contractors to submit cost or pricing data to support price proposals and to certify that the 
data submitted were current, accurate, and complete. By requiring certified data, TfNA tries 
to put the Government on equal informational footing with contractors. 

To insure that contractors are submitting data that are current, accurate, and complete, 
the Federal Acquisition Regulation (FAR) establishes contractor estimating systems review 
requirements. It directs that: 

Cognizant audit activities, when it is appropriate to do so, shall establish and 
manage regular programs for reviewing selected contractors’ estimating 
systems or methods, in order to (1) reduce the scope of reviews to be 
performed on individual proposals, (2) expedite the negotiation process, and 
(3) increase the reliability of proposals (FAR, 1996: 15.81 1). 



7 



The auditor is to send a copy of the survey report to each PCO and ACO having substantial 
business with that contractor (FAR, 1996: 15.81 1). When the auditor is determining the 
acceptability of a contractor’s estimating system, he or she should consider: 



1 . The source of data for estimates and the procedures for ensuring that the 
data are accurate, complete, and current. 

2. The documentation developed and maintained in support of the estimate. 

3. The assignment of responsibilities for originating, reviewing, and 
approving estimates. 

4. The procedures followed for developing estimates for direct and indirect 
cost elements. 

5. The extent of coordination and communication between organizational 
elements responsible for the estimate. 

6. Management support, including estimate approval, establishment of 
controls, and training programs. (FAR: 15.81 1) 



B. DEFENSE FEDERAL ACQUISITION REGULATION SUPPLEMENT 
CONCERNING ESTIMATING SYSTEMS 

The DFARS defines DoD policy in regards to contractors’ estimating systems by 

establishing applicability, responsibilities, and characteristics of an adequate estimating 

system. This regulation applies to all DoD contractors classified as large. To be considered 

a large contractor, the company has to meet one of the two following criteria: 

1 . In its preceding fiscal year the contractor received DoD prime contracts 
or subcontracts totaling $50 million or more for which certified cost or 
pricing data were required. 
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2 . If in its preceding fiscal year the contractor received DoD prime contracts 
or subcontracts totaling $10 million or more (but less than $50 million) for 
which certified cost or pricing data were required and the contracting officer, 
with concurrence of the ACO, determines it to be in the best interest of the 
Government (e.g., significant estimating problems are believed to exist or the 
contractor’s sales are predominantly Government). (DFARS, 1 996: 215.811) 



The responsibilities of the PCO, ACO, Auditor, and Contractor classified as a large 
business are spelled out in the DFARS in more specific terms than the FAR. The 
responsibility of the PCO under the DFARS is to apply the disclosure, maintenance, and 
review requirements (Cost Estimating Systems Requirement clause) to large businesses 
(DFARS, 1996: 215.81 1-70). The ACO is to determine the adequacy of the system and 
pursue the correction of any deficiencies. Auditors, on behalf of the ACO, are to act as team 
leaders in conducting the systems reviews. Lastly, the Contractor subject to this requirement 
shall do the following: 

(1) Maintain an adequate system. 

(2) Describe its system to the ACO. 

(3) Provide timely notice of changes in the system. 

(4) Correct system deficiencies identified by the ACO. (DFARS, 1996: 

215.811-70) 

DFARS also provides characteristics of adequate estimating systems. These 
characteristics should be considered when the ACO is evaluating the contractor’s estimating 
system. These characteristics are as follows: 

1 . Establishes clear responsibility for preparation, review and approval of 

cost estimates. 

2. Provides a written description of the organization and duties of the 

personnel responsible for preparing, reviewing and approving cost estimates. 
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3. Assures that relevant personnel have sufficient training, experience and 
guidance to perform estimating tasks in accordance with the contractor’s 
established procedures. 

4. Identifies the sources of data and the estimating methods and rationale 
used in developing cost estimating. 

5. Provides for appropriate supervision throughout the estimating process. 

6. Provides for consistent application of estimating techniques. 

7. Provides for detection and timely correction of errors. 

8. Protects against cost duplication and omissions. 

9. Provides for the use of historical experience, including a historical vendor; 
pricing information, where appropriate. 

1 0. Requires use of appropriate analytical methods. 

1 1 . Integrates information available from other management systems, where 
appropriate. 

12. Requires management review including verification that the company’s 
estimating policies, procedures and practices comply with this regulation. 

1 3 . Provides for internal review of and accountability for the adequacy of the 
estimating system, including the comparison of projected results to actual 
results and an analysis of any differences. 

14. Provides procedures to update cost estimates in a timely manner 
throughout the negotiation process. 

15. Addresses responsibility for review and analysis of the reasonableness 
of subcontract prices. (DFARS, 1996: 215.81 1-70) 



C. DEFENSE LOGISTICS AGENCY DIRECTIVES 

Defense Logistics Agency Directive (DLAD) 5000.4 defines specifically the duties 
of the ACO within the DCMC in regards to estimating system deficiencies. The ACO is 
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responsible for five actions including: preparing an outstanding estimating system deficiency 
memorandum, reviewing the contractor’s corrective action plan, monitoring the contractor’s 
corrective action, verifying correction of deficiencies, and notifying the contractor of system 
adequacy (DLAD 5000.4, 1995: chap. 2). 

When the ACO determines that the contractor’s estimating system contains 
deficiencies that require correction based on an estimating systems survey, the ACO will 
prepare a memorandum that does the following: 

1. Lists the cost elements covered. 

2. Identifies all deficiencies (significant or otherwise) that require correction 
(including date initially reported). 

3. Addresses the contractor’s progress in correcting the deficiencies against 
the milestones in its corrective action plan. 

4. Estimates the fiscal risk to the Government resulting from the 
deficiencies. 

5. Discusses action taken during the analysis and negotiation of individual 
proposals impacted by the deficiency to ensure that fair and reasonable prices 
are negotiated. 

6. Documents steps taken by the ACO to ensure that the contractor corrects 
the deficiencies in a timely manner. 

7. Forwards a copy of the DCAA audit report and other pertinent documents 
and correspondence. (DLAD 5000.4, 1995: chap. 2) 



This memorandum will be prepared and forwarded to the DCMC District estimating system 
focal point. The memorandum will be updated at least semiannually to reflect the 
contractor’sprogress (or lack thereof). Updated memoranda are due to the district estimatir^ 
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system focal point on 1 February and 1 August of each year. The primary purpose of the 
memorandum is to keep the District staff informed of the situation and confident that the 
ACO has taken appropriate action to remedy the situation. It also provides DCMC insight 
as to the most common type of deficiency and the cost elements most often affected. The 
memorandum may also be forwarded to buying offices or used as an enclosure to field 
pricing reports (however, this action does not eliminate the need for the impact of the 
deficiency on the particular proposal to be quantified within £iny particular pricing report). 
(DLAD 5000.4, 1995; chap. 2) 

The ACO, with help from DCAA auditors and other specialists, will evaluate the 
contractor’s proposed corrective actions for adequacy and completeness. This evaluation 
should include associated milestone dates for reasonableness and timeliness. The ACO will 
apprise the contractor of the adequacy of the plan in writing. (DLAD 5000.4, 1 995: chap. 2) 
The ACO has a continuing responsibility to monitor the contractor’s progress in correcting 
deficiencies. If the contractor fails to make adequate progress against the established 
milestone dates, the contractor’s estimating system should be disapproved in part or in whole 
by the ACO. The ACO shall take whatever action is necessary to ensure that the contractor 
corrects the deficiencies. (DLAD 5000.4, 1995: chap. 2) 

Upon receipt of notification from the contractor that the deficiencies have been 
corrected, the ACO will verily that the deficiencies have been eliminated. When the 
deficiencies have been eliminated, the contractor should be notified that its estimating system 
is considered adequate (and if applicable, withdrawal of estimating system disapproval). 
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This notice will also be sent to the DCAA auditor, and the PCOs doing substantial business 
with the contractor. (DLAD 5000.4, 1995: chap.2) 

D. DEFENSE CONTRACT AUDIT AGENCY PROCEDURES 

DCAA’s general audit policy is that the auditor is responsible for evaluating 
contractor estimating methods and procedures. This is accomplished by estimating system 
surveys designed to evaluate the reliability of a contractor’s estimates, to identity areas 
requiring special emphasis in the audit and negotiation of individual price proposals, and to 
recommend improvements. Estimating system surveys provide knowledge of the strengths 
and weaknesses in a contractor’s estimating system. (CAM, 1995:p. 5118) It is DoD policy 
that contractor estimating systems are reviewed every three years unless past experience and 
current audit risk is considered to be low (DFARS, 1995: 215.81 1-70). 

In addition to meeting the DFARS requirement for review, an estimating system 
survey serves as the fundamental basis for determining audit risk and scope for future price 
proposal and defective pricing audits within DCAA. The primary objectives of the survey 
are to accomplish the following: 

1. Evaluate the adequacy of a contractor’s system for developing cost 
estimates for price proposal purposes. 

2. Evaluate a contractor’s compliance with its written estimating procedures 
and disclosed estimating system (if applicable). 

3. Identify areas of a contractor’s estimating system requiring special 
emphasis or attention during the audit and negotiation of individual price 
proposals. 
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4. Inform interested government activities of the reliability of a contractor’s 

estimating system, and of actions necessary to correct existing deficiencies. 

(CAM, 1995: p. 5120) 

The DCAA estimating systems survey is to examine the following areas: 

1. Internal Audits. The contractor’s estimating system should be subjected to 
periodic internal audits to ensure that the company’s policies and procedures are being 
followed and comply with applicable rules and regulations, and that prior audit findings have 
been corrected. The objective is to recognize the contractor’s commitment to self- 
governance. Contractors may comply with this standard by using in-house efforts or 
through the services of outside consultants. (CAM, 1995: p. 5124) 

2. System Description. The contractor should provide a written description of its 
estimating system in enough detail to allow the auditor to get a thorough understanding of 
how the system operates. This write-up should include the overall organization, assignment 
of responsibilities, and flow of information between each process and function within the 
system as well as other management system. (CAM, 1995: p. 5124) 

3. Training. Training should be provided to appropriate employees on Government 
regulations and company policies and procedures. It should include use of statistical aids 
and advanced estimating techniques. (CAM, 1995: p. 5127) 

4. Cost Estimate Development. A major part of the survey is an evaluation of the 
effectiveness of the methods and procedures used to develop estimates of individual cost 
elements. A contractor should ensure that estimates of source data are appropriately applied 
and the basis for estimating each cost element is provided by written guidelines for 
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developing consistent proposals. Procedures for documenting proposals should cover the 
assumptions used, the basis of each cost element, and the milestones for contract 
performance. (CAM, 1995: p. 5127) 

5. Estimating Methods. The appropriateness of estimating methods used by the 
contractor in preparing proposals should be evaluated. Auditors should examine the degree 
that factors such as the nature of the product or service, the firmness of specifications, and 
the prior experience with related products or services have been taken into account in the 
proposals. A contractor’s estimating system should clearly identify its pricing policy. The 
estimating systems survey should examine the extent that estimating methods make 
appropriate use of historical cost data. Analytical methods should be used, when appropriate, 
to develop cost estimates or evaluate the reasonableness of cost estimates developed using 
other procedures. 

In the area of spare parts and contract changes, the reasonableness of formula pricing 
methods are to be reviewed to ensure individual elements of costs are not duplicated in base 
cost and loading factors. Additionally, the suitability of catalog pricing and prepriced listing 
methods for developing reasonable prices for spare parts proposals needs to be evaluated. 
Acceptability of the contractor’s methods for developing cost estimates for contract changes 
must be assessed. (CAM, 1995: p. 5128) 

6. Proposal Format and Support. Proposal format and support should be evaluated 
to determine that proposals are submitted on appropriate forms (or acceptable substitutes) 
and all data required are furnished. An appraisal of detail contained in proposals and 
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supporting data will be conducted. In addition, a review of the adequacy of contractor’s 
identification of cost and pricing data submitted in support of the proposal and the related 
Pricing Certification should also be examined. (CAM, 1995: p. 5129) 

7. Subcontract Price/Cost Analysis. The contractor’s policies and procedures should 
require that a price analysis be performed for all subcontracts that require submission of cost 
or pricing data. When the contractor is unable to perform an adequate price analysis, a cost 
analysis should be submitted. The results of such analyses should be included as part of the 
contractor’s cost/price analysis and are critical to the negotiation of fair and reasonable prime 
contract prices. For this reason, contractors should have policies and procedures in place to 
accomplish such analyses prior to the submission of their own cost or pricing data. (CAM, 
1995: p. 5129) 

8 . Control and Review of Estimates. Survey the contractor’ s estimating controls and 
reviews to determine the sufficiency of internal controls. This review of controls is to ensure 
uniformity of approach, timely detection and correction of errors, and prevention of cost 
duplications and omissions. The contractor’s level of supervision throughout the estimating 
process impacts the soundness of judgmental estimates and adherence to established 
procedures. The quality, frequency, scope, and results of management reviews of the 
estimating function should be noted. (CAM, 1995: p. 5129) 

9. Contract Certification. The contractor should have established policies and 
procedures for ensuring that all cost or pricing data are current, accurate, and complete as of 
the date of agreement on price (CAM, 1995: p. 5130). 
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E. 



SUMMARY 



DoD has provided a system of regulations to insure that contractors are submitting 
proposals that will place the contractor and the Government on an equal informational 
footing. The AGO is responsible for determiningthe adequacy of the contractor’sestimating 
system. DCAA auditors actually conduct the reviews called estimating system surveys. 
Once completed, the AGO is responsible to insure any deficiencies are corrected. Although 
the regulations and process is rather straightforward, the next Ghapter will show that many 
problems can and do exist in the cost estimating systems arena. 
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111. COST ESTIMATING SYSTEMS 



A. OVERVIEW OF COST ESTIMATING 

Cost estimating requires the application of analysis and experienced judgment in 
projecting material, labor, and subcontract requirements to complete a project on time and 
within budget. Timing constraints and the availability of historical data have an impact on 
the quality of the estimate. Selection of appropriate estimating techniques requires extensive 
analysis by the contractor. Appropriateness of selected estimating techniques should be 
reviewed periodically by the contractor. The techniques used when the program is in the 
concept exploration stage, or when no bill of materials exists, are usually not the techniques 
that would be appropriate for programs in the production and fielding stage. Because cost 
estimating integrates technical as well as financial information, the process requires input 
from every organizational element of the business. (CAM, 1995: p. D16) 

Although contractor estimating systems differ in approach and philosophy, their basic 
objectives are the same. Cost estimates are a series of informed projections and assumptions 
based on available information existing at the time of proposal preparation. (CAM, 1995: p. 
D16) Cost estimating is series of logical steps. The level of detail required in each step is 
affected by the contract requirements expressed in the RFP. Typical steps in cost estimatii^ 
include the following: 

1 . Ensuring that all relevant background documents such as historical costs, 
drawings, and specifications are available to assist in understanding job 
requirements. 
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2. Determining which estimating techniques will be used, the level of detail 
required, and the amount of time available to generate and document a 
completed estimate. 

3. Determining if quotes and other information will be required from outside 
sources. 

4. Deciding if any elements require further clarification, redesign, or have 
potential manufacturing difficulties. 

5. Determining if the capability and capacity to manufacture required 
components exist in-house. 

6. Determining if further information is required to develop and complete 
estimates. 

7. Coordinating the activities of departments participating in the estimating 
exercise. 

8. Obtaining quotes, history, and other bases for material and subcontract 
items. 

9. Assembling direct costs by cost element, and computing indirect expenses 
using appropriate factors and rates. 

1 0. Consolidating proposal elements and documenting preparation rationale. 

(CAM, 1995:p. D16) 

B. TYPES OF COST ESTIMATING 

The types of cost estimating methods selected are dictated by the availability of 

historical data, the nature of product or service, and the Government requirements. Cost 

estimating methods may be categorized into six main groups: subjective, parametric, 

comparative, synthetic, global, and research and development (R&D): 

1. Subjective. This estimating method develops costs using 
experience, judgment, memory, informal notes, and other readily available 
data. Typically, these kinds of estimates are used in proposals when a 
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drawing has not yet been developed or the contractor is faced with limited 
proposal preparation time. 

2. Parametric. This method creates labor and material estimates by 
statistically analyzing and manipulating historical data to reflect current 
quantity requirements. For example, previous raw material requirements on 
a price per pound basis could be used to project current proposal amounts. 
Parametricsuse one or more cost estimating relationships (CER) to estimate 
costs associated with the development, manufacture, or modification of an 
end item. Special cost comparisons are required to validate parametric 
estimating systems. Variables used in CERs must be logically related and 
statistically valid. The rationale for selecting the variables should be well 
documented. Parametricsare often used to cross-check estimates developed 
using other estimating techniques. 

3. Comparative. This method develops proposed costs using like 
items produced in the past as a surrogate. Allowances are made for product 
dissimilaritiesand changes in complexity, scale, design, and materials. The 
comparative method can be used in conjunction with parametric estimating 
and can be used to develop adjusted unit costs while parametrics are applied 
to projectthe newly proposed quantities. Improvementcurve applications are 
an example of comparative estimating. 

4. Synthetic. This method divides proposals into their smallest 
component tasks. Estimates are developed for component tasks which make 
up the whole. Synthetic estimates are normally supported by detailed bills 
of material. 

5. Global. This is a quick and subjective technique used to determine 
the advisability of continuing with a project. 

6. Research and Development (R&D). There are two basic 
approaches available for the R&D type of estimating. The first is a simple 
form of targeting R&D objectives in the context of a fixed budget. As in the 
preparation of routine budgets, the breakdown should be compatible with the 
cost accounting system and procedures established to monitor and control 
expenditures. A second method of estimating R&D is a trial and error 
procedure involving an interchange of ideas and information including all 
available records of past R&D effort and experience. Because there are so 
many unknown factors involved in R&D effort, the potential for error in this 
type of estimating is especially great. (CAM, 1995: p. D18) 
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c. 



ESTIMATING PROCESS AT A TYPICAL CONTRACTOR 



Although each contractor has their own estimating system that fits their business, the 
following is an example of a typical contractor. Generally at large contractors, the estimating 
(or pricing) department usually has overall responsibility for coordinating and assembling 
estimates to be incorporated into proposals authorized by top management. Preparation of 
detailed estimates is accomplished by the departments that will actually perform or supervise 
the work if the contract is received. (CAM, 1995: p. D16) 

The cost estimating venture is usually initiated in response to an RFP. The RFP 
provides a statement of work (SOW) that outlines the Government’s requirements. It is also 
a source of information in establishing a baseline for labor and material requirements. The 
contractor’s proposal should include tasks and materials consistent with the SOW. When 
top management authorizes a response to an RFP, the estimating department reviews the 
SOW and any top-management guidance and issues a cost estimate request to other 
departments within the company that will be involved in the proposal. The estimating 
department generally has primary responsibility for coordinating the overall submission and 
authorizing the finalized proposal. (CAM, 1995: p. D16) 

Contractors may also submit unsolicited proposals that have no outstanding RFPs. 
When such proposals are pursued by a buying activity, the PCO will normally request a more 
detailed cost proposal before requesting an audit. The estimating process should be the same 
as when there is an RFP. (CAM, 1995: p. D17) 
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When production is necessary on items not previously produced, the estimating 
department (or the related project management department) solicits a preliminary conceptual 
design from the engineering department. The preliminary design should be detailed enough 
so that individual parts can be identified and numbered. After the preliminary design has 
been completed and reviewed, a work breakdown structure (WBS) is prepared. The WBS 
is a matrix that organizes and describes proposed tasks and identifies the performing 
company unit. This is best done before the details of the cost estimate request are finalized. 
(CAM, 1995:p. D17) 

The planning process includes the preparation of delivery schedules, staffing 
projections, time-span requirements, and funding estimates. Planning is a cooperative effort 
that involves the estimating, engineering, and production control departments. (CAM, 1995; 
p. D17) 

“Grass-roots estimates” are basic estimates of labor, material, and other direct costs 
developed by the individual business units that will actually perform the work. In some 
cases, departments or divisions are asked to generate price estimates. When this occurs, 
special care must be used to ensure that sound purchasing considerations such as competition 
and quantity discounts are applied to the estimates. (CAM, 1995; p. D17) 

The engineering department usually develops labor hour estimates for any potential 
items to be produced in-house. These estimates are normally prepared at the individual part 
level. The manufacturing department uses this information along with historical data to 
project labor requirements. These projections may be broken down by functional area and 
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cost center. A variety of techniques including manpower loading, statistical relationships, 
past experience, and judgment are used to produce the estimates. Additional information 
such as program schedules and configuration/performance characteristics from preliminary 
and final engineering design drawings may be worked into the estimates. In all cases, the 
methods used to produce direct labor estimates should be discernible, and supporting 
documentation should be available for verification. (CAM, 1995: p. D17) 

A make-or-buy committee reviews required materials and associated labor, and 
determines which items should be produced internally. In some instances, decisions will be 
deferred until a contract award is made and further design effort completed. (CAM, 1995: 
p. D17) 

The estimating department requests purchasing to provide estimates for all potential 
buy items. The purchasing department is provided with the best available specification data 
from engineering. Delivery requirements are provided by production control. Material imit 
prices (including purchased parts, raw material, buy to drawing items, and subcontract items) 
are obtained by the purchasing department from vendor quotations, current purchase orders, 
catalogs, and in some cases statistical methods. Material costs are usually developed by 
applying these prices to unit quantities in a bill of materials provided by the engineering or 
manufacturing department. The purchasing and estimating departments are usually 
responsible for determining appropriate material escalation factors. Escalation is either 
quoted by major vendors or projected using specific price indices. (CAM, 1995: p. D18) 
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Each estimate is reviewed and approved at the functional level. These estimates are 
then submitted to the estimating department that assembles the complete proposal. 
Estimating personnel integrate, adjust, and analyze estimates for accuracy and completeness 
The cost estimate is summarized further by functional organization, major tasks, and other 
breakdowns required by the RFP. When all direct cost elements have been received and 
properly classified, applicable direct labor rates and indirect expense rates are applied to 
complete the basic cost estimate. These rates may be developed by either the estimating or 
accounting departments. Fee calculations are usually applied in accordance with RFP 
guidance and company pricing policy. (CAM, 1995: p. D18) 

Subsequent to initial pricing and the determination of profit factors, the proposal is 
reviewed by a management committee usually consisting of representatives from marketing, 
accounting, plant management, estimating, and the program office. The committee 
scrutinizes the reasonableness of estimates, overall acceptability, and compatibility with the 
company’s business strategy. This process is completed by the formal release of the pricing 
proposal and supporting rationale. (CAM, 1995: p. D18) 

D. PROBLEMS IDENTIFIED WITH CONTRACTOR ESTIMATING SYSTEMS 

Numerous studies conducted by the GAO and others between 1 987 and 1 994 have 
focused on defective pricing and inadequate cost estimating systems. Most of these reports 
detail that problems still exist in this area. (GAO 93-8) The area that poses the most 
potential risk to the Government is subcontract management. The majority of work on major 
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DoD programs is increasingly accomplished by subcontractors. This has not always been 
the case, as programs in the 1950's subcontracted out as little as nine percent of total 
procurement funds, vice the 50 to 75 percent seen today (Mooney, 1991; p. 12). Figure 1 
illustrates the growth in subcontracting by DoD prime contractors. 

Growth In Subcontracting 




I Direct Labor H Subcontracting 

Figure 1. Source: Mooney, 1991 

The first situation that these studies have identified as a problem area in cost 
estimates is that prime contractors obtain price reductions on competitive subcontracts after 
negotiation with DoD. The DoD requirement for a contractor to obtain cost or pricing data 
supporting subcontractor proposals, to evaluate those data, and to provide the evaluation 
results to the Government as part of their proposals does not apply to subcontracts awarded 
on the basis of adequate price competition because competition is presumed to produce fair 
and reasonable prices. (GAO 93-8, 1992: p. 15) This can be seen in the following example: 
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GAO reviewed 66 subcontracts worth $44 million whose estimates 
were identified by a contractor as having been competitively obtained. GAO 
found that after “competitively” soliciting subcontract prices to support its 
proposals to the Government, the contractor had resolicited prices and, on 55 
of the subcontracts, obtained prices that were $10.4 million lower than what 
was proposed and included in the prime contracts. Conversely, the contractor 
awarded 10 subcontracts at prices that were about $1.5 million more than 
what was proposed and negotiated in the prime contracts. Thus, the prime 
contractor’s actions subsequent to negotiating with the Government resulted 
in a net amount of $8.9 million less than agreed to with the Government. 
(GAO/HR 93-8, 1992: p. 16) 



Prime contractors awarding noncompetitive subcontracts at a cost lower than 
proposed is the second situation involving subcontracting that GAO identified as a problem. 
In addition to negotiating lower prices on competitive subcontracts, some prime contractors 
negotiate substantial price reductions on noncompetitive subcontract proposals after 
completing negotiations with the Government (GAO/HR 93-8, 1992: p. 16). 

... examined 12 noncompetitive subcontract estimates, each in excess 
of $ 1 million. They found that in the aggregate, prime contractors had made 
awards on these estimates for about $8.8 million less than the price 
negotiated in the contracts with DoD. (GAO/HR 93-8, 1992: p. 17) 

DoD must have an effective oversight program in addition to requiring sound 
contractor cost estimating systems. This Governmental oversight program should do two 
things. First, it should identify and audit contractors that are considered high risk for 
overpricing contracts. Secondly, this program should ensure timely and effective corrective 
actions when deficiencies are found. 

DCAA is the principal agency responsible for conducting defective pricing audits to 
determine whether contractors have complied with TINA. Because of the large number of 



27 



contracts and subcontracts that are subject to audit, DCAA cannot audit all contracts and 



subcontractors subject to TINA. As a result, DCAA allocates its audit resources based on 
its assessment of risk. In assessing risk, DCAA considers, among other factors, the adequacy 
of contractors’ cost estimating systems, accounting systems, and their histories of defective 
pricing. (GAO/HR 93-8, 1992; p. 18) 

For example, high-risk, fixed-price contracts worth less than $10 million 
each, DCAA’s selection criterion is for it to audit 1 in 15 contracts. For 
medium-high risk, fixed-price contracts, its selection plan calls for it to audit 
1 in 30 contracts worth less than $10 million each. (Richards, 1996: p. 23) 



In the past, DCAA has not been able to complete all the audits that its plans call for 
(GAO/HR 93-8, 1992: p. 19). The majority of both DCAA auditors and contractors agree 
that the level of auditing pertaining to subcontract management oversight has decreased. 
Much of this is related to the downsizing of DoD and a resultant shrinking workforce. 
(Beaubien, 1995: p. 114) In order to allocate its resources to contracts with the highest risk, 
DCAA must be aware of the universe of both prime and subcontracts subject to audit under 
TINA (GAO/HR 93-8, 1995: p. 20). This is not always accomplished. 

In reviewing a sample of 2 1 1 negotiated subcontracts having a dollar 
value of about $337 million, GAO found that DCAA was not aware of 1 86, 
or 88 percent, of the subcontracts. The 186 subcontracts were about $189 
million, or 56 percent of the total value of subcontracts in our sample. 
(GAO/HR 93-8, 1992: p. 20) 



The second factor for a quality oversight program is to ensure effective and timely 
corrective action once deficiencies are noted. This can only be done with an effective 
follow-up system. 
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We reviewed DoD’s audit follow-up system and found that the 
system did not provide accurate and complete information on the condition 
of cost estimating systems for many high risk contractors. The follow-up 
system data was inadequate in three areas. The system (1) was missing 
reports on contractors; (2) understated the length of time cost estimating 
deficiencies remained uncorrected; and (3) showed contractors had corrected 
deficiencies, when our review showed they had not. (GAO/HR 93-8, 1992: 
p. 23) 



Timeliness in correcting deficiencies is also a problem. In 1991, GAO reported that 

many cost estimating deficiencies had remained uncorrectedfor long periods of time despite, 

the 1 988 revision in DoD’s regulation requiring major contractors to establish adequate cost 

estimating systems and DCAA audits reports to identify estimating deficiencies 

(GAO/NSIAD91-157, 1991:p. 7). Toilluminatethispoint,in 1994 GAO auditors reviewed 

the status of 30 contractors identified as having high risk cost estimating as of May 1992. 

As of January 1994, the 30 contractors had corrected or potentially corrected 
85 of the 117 significant deficiencies. However, the 32 remaining 
deficiencies had been outstanding an average of 3.8 years. (GAO/NSIAD 94- 
153, 1994: p. 3) 



The results of not having an effective contractor cost estimating and Governmental oversight 
program costs the Government additional resources. These resources are both monetary 
(higher contract prices) and time. The extra steps DoD PCOs take are frequently time- 
consuming and costly. 

For example, in December 1991, DCAA reported that a certain 
contractor had five significant estimating system deficiencies. DCAA first 
reported these deficiencies in September 1990. The deficiencies remained 
uncorrected when, in June 1992, the contractor submitted a $550 million 
proposal to DOD that was based on costs generated by the estimating system. 
(GAO/NSIAD 94-153, 1994: p. 5) 
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DCAA and Army personnel involved in the negotiation identified numerous errors in the 
proposal that would have significantly increased the cost. 

DCAA’s audit of the proposal found it to be unacceptable as a basis 
for negotiating a fair and reasonable price. When the contractor resubmitted 
data, DCAA found much of it still unacceptable. (GAO/NSIAD 94-153, 

1994: p. 5) 

This extra probing of the contractor’s proposal was time-consuming because the 
Army was not sure of the quality of the data submitted and needed to verify nearly all the 
information in the proposal. As a result, DCAA used more audit resources than it normally 
would have. (GAO/NSIAD 94-153, 1994: p. 5) Army contracting officials invested 
considerable effort in obtaining and reviewing information from the contractor. The Army’s 
Contracting Officer submitted about 100 requests to the contractor for additional 
information, more than twice the normal number of requests. Army contracting officials said 
that “because they did not trust the contractor’s estimating system, reviewing the data the 
contractor provided was time-consuming” (GAO/NSIAD 94-153, 1994: p. 5). 

E. SUMMARY 

Although the majority of actions associated with contractor estimating systems are 
conducted by the ACO and DCAA auditors, the PCO is significantly impacted by these 
systems. As illustrated above, the inadequacies of contractors’ estimating systems cost 
PCOs in funding and time. Chapter IV will present the survey questionnaires sent to PCOs, 
the responses received, and the researcher’s analysis. 
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IV. SURVEY DATA PRESENTATION AND ANALYSIS 



A. INTRODUCTION 

This segment of the research effort is centered on determining the specific actions 
PCOs take when notified of contractor estimating system deficiencies or disapprovals. The 
goal was to obtain factual data and opinions from PCOs regarding how they received 
notificationand what actions they took once notified. To achieve this goal, a survey was sent 
to PCOs with the most probable exposure to deficient contractor estimating systems. 

Surveys were mailed in early September to 272 PCOs at 57 different buying 
commands. A total of 106 responses, or 39%, were returned representing 40 of the 57 
commands. These 40 commands consisted of 20 Air Force, ten Navy, eight Army, and two 
DOD agencies. The 106 surveys were received from 32 Air Force PCOs, 41 Navy PCO’s, 
3 1 Army PCO’s, and two from DoD agency PCOs. Anonymity for the individual respondCT 
was guaranteed in an effort to ensure that PCOs would respond to the questionnaires 
completely and honestly. Therefore, the researcher did not attempt to isolate or identify any 
individual responder during analysis of the data however, individual commands were 
identified. Appendix A provides a complete copy of the survey questionnaire utilized for 
this research effort. Appendix B provides the aggregate responses by command. 

The researcher used a judgment sample to conduct the survey. A judgment sample 
targets those most closely related to the issue being investigated. Since each member of the 
population does not have an equal chance of being chosen, this is a non-random sampling 
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method. The reason for using a judgment sample was to survey the most probable PCOs 
with exposure to estimating system deficiencies zind disapprovals. The judgment sample 
population is comprised of PCOs for DoD acquisitionprograms with RDT&E budgets of $25 
million or greater. The researcher’s assumptions about the sample population are: (1) the 
responder will have a high experience level, and (2) the value of contracts the responding 
PCOs are responsible for are large enough to warrant review of contractors’ estimating 
systems. 

B. SURVEY COMPOSITION 

The survey questionnaire consisted of nine questions that were designed to obtain 
both objective and subjective responses concerning contractor cost estimating system 
deficiencies and disapprovals with respect to PCO actions once notified. Some questions 
were of the “open ended” type. These questions were included to gather data on why PCOs 
were notified and what action they took once notified. Additionally, several demographic 
questions were posed to obtain data concerning the individual respondent. Demographic 
information was requested to provide experience level and contract value validity of the 
survey results. 

The survey was limited to one page and the researcher estimated that it would require 
approximately ten minutes to complete the questionnaire. The researcher felt that a more 
concise survey would increase the response rate. The survey asked the responder to answer 
questions 5, 6, and 7 for the three most recent occurrences of disapproved or deficiency 
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notification. This was done to maximize the available data for analysis. The researcher felt 
that PCOs in the sample would have had multiple experiences with deficient estimating 
systems. This was due to the bias of the sample population surveyed. Allowing them 
multiple responses to these questions would provide more information for the purpose of 
analysis. 

C. SURVEY RESPONSES AND ANALYSIS 

Each question from the survey is listed below, followed by a summary of the answers 
received and an analysis. 

Question 1. How many years of contracting experience do you have? 

a. 0-3 years 

b. 4-8 years 

c. 9-15 years 

d. 16-25 years 

e. 26 years or greater 
Responses: The survey results were: 

2% 0-3 years 

3% 4-8 years 

20% 9-16 years 

58% 16-25 years 

1 7% 26 years or greater 



33 



Of the 106 participants responding, 95% had nine or more years of experience. The 
most frequent response from all Services was 16-25 years of experience as shown in Figure 
2 . 



Years of Experience 



70 



60 

50 

40 

30 

20 

10 

0 




[ ] 0-3 years 

0 4-8years 
f/H 9-15 years 

1 16-25 years 
L] 26 or greater 



Source: Developed by researcher 



Figure 2 



Analysis: Question 1 was asked in order to acquire demographic information 
regarding the sample population. It is intended to be used later to form conclusions about 
responses to other survey questions. The high experience level of the participants indicates 
that these PCOs could have potentially worked on several proposals produced by a 
contractor’s cost estimating system throughout their career. This level of experience should 
not be surprising. PCOs in this population of large programs, generally have many years of 
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experience before being entrusted with this degree of responsibility. It should also be noted 
that this distribution of experience does not necessarily mirror the experience levels of the 
entire PCO population. 

The distribution of the responses from each Service is roughly the same as that 
depicted in Figure 2. Five respondents had eight years or less experience. Two respondents 
from the Air Force, two from the Navy, and one from the Army fell into this category. All 
five were from different commands. No individual command displayed characteristics that 
would distinguish them from the “norm.” 

Question 2. What is the average dollar value of contracts you are responsible for per 

year? 

a. 0-$l million 

b. $1-$10 million 

c. $10-$50 million 

d. $50-$ 100 million 

e. $100 million or greater 
Responses: Survey results were: 

7% 0-$l million 

9% $l-$ 10 million 

19% $10-$50 million 

17% $50-$l 00 million 

48% $100 million or greater 
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Eighty-four percent of the respondents were responsible for contracts worth $10 
million or more per year. Figure 3 shows the most frequent response to the question was in 
the range of $100 million or greater. 



Dollar Value of Contracts 
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Source: Developed by researcher 



Figure 3 

Analysis: Once again this question was asked to gather demographic information 
about the sample population. It is intended to be used later to form conclusions about 
responses to other survey questions. 

Contractors’ estimating systems are subject to Government review when the 
preceding year’s business with DoD is $50 million or more, or $10 million or more when 



certified cost or pricing data are required. (DFARS, 1996:215.811-70) Figure 3 shows that 
the majority of the PCOs surveyed conduct business well over this threshold. This indicates 
that the PCOs sampled will have a high probability of dealing with contractors with 
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estimating systems that are subject to Government review. The survey results support the 
researcher’s second assumption about the sample members; that is, the contract value is large 
enough to be subject to Government oversight. Again it should be noted that this distribution 
does not necessarily mirror the entire PCO population. 

The distribution of the responses from each Service is roughly the same as the 
aggregate depicted in Figure 3. Three individual commands appear to comprise an abnormal 
share of the more than $ 1 00 million category. The Army Missile Command (MICOM), the 
Naval Air Systems Command (NAVAIR), and the Naval Sea Systems Command 
(NAVSEA) account for 23 of the 5 1 (45%) responses in the greater than $ 1 00 million range. 
The other 55% of this category is made up of 20 commands. This indicates that, for the 
purposes of this survey, most of the dollars are spent in the three aforementioned commands 

Question 3. Have you ever received notification of a contractor’s estimating system 
having deficiencies or being disapproved? 

a. Yes 

b. No 

Responses: Survey results are: 

69% Yes 
31% No 

Seventy -three of the respondents stated they had received notification of a contractor 
having deficiencies or disapproved estimating systems. Figure 4 shows these results. 
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Analysis: This question presents a fork in the road for the rest of the research. PCOs 
that responded with a “no” answer were not required to complete any further questions on 
the survey. Since the results of the survey question are based on a judgment sample, readers 
should not conclude that 69% of all PCOs are dealing with contractors who have deficient 
estimating systems. This percentage only represents the sample population. The validation 
of the judgment sample method is achieved by the results of question 3. Surveying a 
population whose members have a great deal of experience (nine or more years) and are 
conducting a large amount of business ($10 million or more) yielded the most probable 



candidates who have dealt with deficient estimating systems. 
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Figure 4. 

Using a weighted average from the data gathered in questions 1 and 2, PCOs that had 
not been notified of deficiencies had more experience and were responsible for higher dollar 
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value contracts. Although no distinguishing characteristics were noted causing this group 
to be different from the group answering “yes,” the industry that the PCO deals in may be 
the explanation for the above situation. Among the PCOs that were notified of deficiencies, 
the years of experience seems to be the more dominant variable for notification. This 
situation of dominance occurs in all three Services and across a variety of industries. 

The responses varied seven percent between the Services. The Air Force had the 
highest percentage of “yes” answers (72%) while the Army had the least (65%). In the 
opinion of the researcher this variation is not significant. Three commands had a large 
impact on the 69% overall notification rate. MICOM, NAVAIR, and NAVSEA had a 76% 
notification rate while the other 30 commands, within the sample of “yes” respondents, 
equated to 64%. 

The grouping of “no” responses had no distinguishing characteristics. They were 
from all three Services, representing 1 8 commands. These commands included eight Air 
Force, five Army, four Navy, and one DoD agency. The PCOs in this group had varied 
experience levels, and were responsible for varied dollar contract values. 

An interesting situation surfaced when comparing the “yes” and “no” responses by 
command. Ten of the 40 commands that sent back surveys had some PCOs responding 
“yes” and some responding “no”. Within these ten commands, 43 PCOs responded “yes” 
and 21 responded “no”. The two populations of “yes” and “no” responders are responsible 
for roughly the same dollar value of contracts. This situation occurs in all three Services 
with four commands from the Air Force, three from the Navy, and three from the Army. 
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This would not be surprising if the commands bought totally unrelated products with a wide 
supplier base; however, this may not be the case. Many of the markets in which these PCOs 
participate have few suppliers. This information by itself is not enough to draw any 
conclusions. 

A PCO may give a “no” response because he or she was never officially notified by 
the ACO or DCAA auditor of any problems. They may have learned from other PCOs or 
may have found out through other methods that a contractor had estimating system problems. 
If this was the case and they took actions to protect the Government from being overcharged, 
this survey would not have obtained that information. This should be considered a weakness 
of the survey questionnaire. 

Question 4: Why were you notified? 

Responses: This was an open-ended question asked to determine the relationship 
between the PCO and the Contractor. The returned comments were varied but the researcher 
grouped them into four general categories. These categories are: ( 1 ) the result of a proposal 
evaluation request, (2) the PCO is on a standard distribution list or the PCO of record for the 
program, (3) miscellaneous answers, and (4) no response. The grouping that responders 
identified most often (5 1 %) was the proposal evaluation category. Figure 5 details how the 
73 responses are distributed. 

Two of the respondents answered the question in a way that did not place them in any 
of the above categories. One of these two discussed cost estimating relationships of a 
specific contractor. The other discussed the fact that the contractor they were dealing with 
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did not have an estimating system at all. Neither respondent answered the question of why 
they were notified. These two surveys were considered “non-responsive and have been 
included in the ‘‘no response category.” 

Analysis: The most frequent reason a PCO was notified about a deficient estimating 
system was an ensuing negotiation. This situation is caused by DoD’s policy stating that 
PCOs shall request Field Pricing Reports for: 
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Why PCO was Notified 
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Figure 5 



1. Fixed-Price Proposals Exceeding $500,000. 



2. Cost-type proposals exceeding $500,000 from offerors with significant 
estimating deficiencies. 



3. Cost -type proposals exceeding $10 million from offerors without 
significant estimating deficiencies. (DFARS, 1996: 215.805-5) 

The requirement for a field pricing report is the driver for the high response in this category. 
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Informing PCOs of a contractor’s estimating system adequacy on a proposal-by- 
proposal basis provides the most accurate assessment. ACOs are to semiarmually update the 
status of the contractor’s deficient system via a memorandum to their DCMC District office. 
The memorandum may also be enclosed in the field pricing report. PCOs using the last 
complete estimating system survey as a basis for decisions can potentially be working with 
two to three year old information. 

The second most frequent response was a result of being on a standard distribution 
list or being the PCO of record for the program. The PCOs in this grouping are either 
currently in performance of a contract with the contractor or have conducted a significant 
amount of business with the contractor. PCOs in this group are interested in the contractor’s 
estimating system for potential defective pricing and contract modification purposes. 

The third grouping of responses was categorized as miscellaneous. The comments 
received indicated that the PCO wanted to maintain a good working relationship with the 
ACO and that generally PCOs are notified because businesses have merged. As expected, 
the low number of responses falling into the miscellaneous category was not surprising. 
Notification tends to come from the usual source of proposal evaluations or from being the 
PCO of record for an ongoing contract. 

The fourth group had no response. It included the two surveys listed above that did 
not give a logical answer for the question and one left unanswered. 

The variation among responses of the three Services was not significant. Army PCOs 
had the highest percent of responses in the proposal evaluation category with 56%, while 
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Navy PCOs had the lowest at 46%. None of the commands responding had any outstanding 
characteristics. 

Question 5: How did you learn of the deficiencies or disapprovals? 

a. ACO 

b. DCAA 

c. Contractor 

d. Cost Analyst 

e. Govt. Tech/Eng 

f. Other 

Response: Survey results were: 



42% 


ACO 


49% 


DCAA 


5% 


Contractor 


3% 


Cost Analyst 


0% 


Govt. Tech/Eng 


1% 


other 



Questions 5, 6 , and 7 ask for information about the three most recent occurrences of 
notification. With 73 surveys returned answering that they had been notified, there are a total 
of 219 responses possible. Of the 219, at least 151 occurrences were reported for each of 
these questions. The majority (125) of the PCO’s responses identified only one source of 
notification, while 22 PCO’s responses identified two sources and four PCO’s responses 
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identified three or more sources of notification. Figure 6 shows that the vast majority of the 
time (91%) the PCOs obtain their information about deficiencies from the ACO or DCAA 
auditors. The other nine percent of the time they acquired it from the contractor, cost 
analysts, or other sources. 



Who notified PCO 
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Figure 6. 



Analysis: The survey results show that PCOs were informed of deficient estimating 
systems by ACOs 42% of the time. The Navy and Air Force identified the ACO as the most 
frequent source with 49% and 48% respectively. The Army, on the other hand, was 
informed by the ACO only 27% of the time. A possible reason for the low Army percentage 
may be that an aircraft carrier costs a lot more and takes a lot longer to build than a hand held 
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radio. Therefore, a contractor building an aircraft carrier would be subject to more review 
by the ACO than the contractor producing the hand held radio. NAVSEA and NAVAIR 
accounted for 80% of the total marking of this response for the Navy. This is not surprising 
considering the large dollars obligated and type of products these two commands buy. No 
specific Air Force command was a driver for these results. 

Notification from DCAA was the most frequent response overall. It accounted for 
49% of all answers. Army PCOs identified this response as the most frequent (65%). This 
high percentage was driven by one command. MICOM accounted for 21 of the Army’s 34 
(62%) responses. These 21 responses represented 12 Army PCOs. Eight of these 12 
identified that they were due to a proposal evaluation. The other 4 were on a standard 
distribution list or the PCO of record. This indicates that MICOM has a particularly good 
communication relationship with DCAA and that the length of a procurement has an affect 
on who notifies the PCO. 

The fact that 91% of the PCOs are notified by the ACO or DCAA confirms that the 
PCOs are being notified by the proper Government personnel as established in the FAR, 
DFARS, and DLA Directives discussed in Chapter II. The fact that these groups are doing 
the majority of notification does not speak to the quality of the information. As pointed out 
in GAO audits, many ACOs are declaring a deficient system “adequate” before it truly is. 
(GAO/HR 93-8, 1992: p. 23) This subject of ACO adequacy determination is an area in 
which fiirther research should be conducted. 
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Contractors were identified as the third most frequent source of notification, at the 
same time these PCOs also identified the ACO. The dual notification was identified in every 
case that the contractor was listed as a source of notification. This indicates that the ACO 
notification may have led the contractor to admit to the problem. Other research has 
identified that contractors are not always the most forthcoming with negative information 
about their estimating systems. Based on a survey of DCAA auditors and contractors, the 
auditors believed that one quarter of contractors had subcontract estimating deficiencies 
while less than two percent of the contractors thought they did. (Beaubien, 1995: p. 112) 

Cost Analysts were identified 3% of the time in the survey results. Four responses 
were received from the Air Force and one from the Navy. Three of the five responses were 
from one Air Force PCO. The intention of the researcher in this category was to identify cost 
analysts within the PCO’s organization or program. Nonetheless, some respondents may 
have answered this question thinking of the cost analysts within the DCMC structure. 
Regardless of any misinterpretation of the meaning of the answers provided, the percent of 
the total is not material. 

None of the PCOs identified the Government Technical and Engineering community 
as a source of information about a contractor’s estimating system. Although the researcher 
included them as an option, the fact that no PCO identified them as a source is not surprising. 
In the researcher’s opinion, this shows that the Technical and Engineering community is 
more interested in what is being proposed than how much the proposal will cost. 
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The “other” option was provided to try to eliminate any bias that would be created 
by providing specific answers to the question. The researcher’s hypothesis was that some 
PCOs would have learned of contractors’ estimating system problems from other PCOs or 
through their contracting chain of command. This did not turn out to be the case. One 
possible reason for the low response in this category is that PCOs may feel self-conscious 
about having to admit that their chain of command had to notify them of deficiencies 
regarding their contractor. 

The “other” category received two responses. The first was that a PCO had 
discovered the deficiency himself and informed the ACO. This instance has to be one that 
slipped through the process, however, the second instance was identified by the Army Audit 
Agency. Based on the other answers by this respondent throughout the rest of the survey, 
this was not the first indication of a problem. 

When examining the responses to this question, the researcher noticed that some 
questionnaires identified multiple sources of notification. This examination revealed that 
125 of the total 151 (83%) identified a single source of the notification. Twenty-six of the 
total 151 (17%) identified multiple sources of notification. This situation shows a weakness 
in the researcher’s questionnaire. The question should have asked the responder to only 
answer one per notification occurrence. The researcher’s intent was to identify from whom 
the PCO first learned of the deficient estimating system. 

Question 6. What actions did you take once notified? 

Responses: Survey results were: 
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62% Internal to Government 
1 7% Direct with Contractor 
16% Both 
5% Miscellaneous 



Action Taken 
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Source: Developed by researcher 
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Figure 7. 

This was an open-ended question asked to determine the actions taken by the PCO. 
The returned comments were varied but the researcher grouped them into four general 
categories identified in Figure 7. The most frequent response fell into the grouping classified 
as Government internal action. Responses in this category include the PCO determinirg the 
impact of the deficiency, the PCO discussing the deficiency with the ACO or DCAA, the 
PCO informing their chain of command, and others that indicated the PCO actions were 
confined to dealings within the Government. The second most common response grouping 
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was direct communications with the contractor. This category consisted of responses that 
indicated the PCO contacted the contractor directly and discussed or resolved the issues 
without other identified Government intervention. The third grouping, labeled as both, 
consists of responses indicating the PCO formed some kind of working or discussicn group. 
The group would consist of, not only the PCO and contractor, but also the ACO and DC AA 
auditors. The last grouping is labeled as miscellaneous and consists of six responses that did 
not fit into the other categories. 

Analysis: This question was included to help answer the primary research question 
of what actions PCOs take when notified of deficient contractor estimating systems. The 
researcher observed from the survey that 95% of the time the PCO took some form of 
positive action. Therefore it can be said that receiving notification of a deficiency in a 
contractor’s estimating system has an impact on the PCO and the majority of the time his or 
her response is internal to the Government. 

A total of 1 68 responses were received for this question. PCOs took actions internal 
to the Government 101 times or 60% of the time. Twenty- six responses (16%) stated that 
the PCOs worked directly with the contractor. Twenty percent of the time the PCOs worked 
as a team with the ACO, the DCAA, and the contractor. There was no significant variation 
of results between the Services for any of these categories. 

Question 6 asked all respondents to identify their actions regardless of whether they 
were in negotiations or not. Question 7 asks of the PCO’s actions if in negotiations. 
Therefore, PCOs answering question 6 that are in negotiations would also answer question 
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7. To unmask the affect of potential double counting, the researcher eliminated all PCOs in 
a negotiation status from question 6. This new grouping consists of only PCOs on a standard 
distribution list or PCOs of record for programs, and miscellaneous and no responses from 
question 4. The actions these PCOs took can be grouped into the same categories identified 
in question 4. Fifty-five percent of them took actions internal to the Government. Eleven 
percent worked directly with the contractor to resolve the issues and 26% worked both with 
the contractor and other Government personnel such as the ACO and DCAA. This 
distribution is roughly the same for all PCOs in the sample population. There is no evidence 
to indicate why this situation occurs. There was no significant variation between the 
Services in this grouping. 

Two of the responses in the miscellaneous category stood out from the others. The 
PCOs in these cases stated that they awarded the contract to the next higher priced 
contractor. From a contractor’s perspective, this action would grab one’s attention quickly. 
The viability of the company may rest with Government contracts. Businesses that can not 
compete because of their estimating systems have significant problems. 

Question 7. If in negotiations, how did you proceed? 

a. No change in plan. 

b. Allowed the contractor additional time to correct the deficiency and 

submit a corrected proposal. 

c. Considered another type of contract. 
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d. Used additional cost analysis to determine the reasonableness of the 
cost elements affected by the system’s deficiency. 

e. Segregated the questionable areas as a cost reimbursable line item. 

f. Reduced the negotiation objective for profit or fee. 

g. Included a contract reopener clause that provides for adjustment of the 
contract amount after award. 

h. Other. 

Responses: Survey results were; 

13% No change in plan. 

25% Allowed the contractor additional time to correct the deficiency and 
submit a corrected proposal. 

2% Considered another type of contract. 

3 1 % Used additional cost analysis to determine the reasonableness of the 
cost elements affected by the system’s deficiency. 

4% Segregated the questionable areas as a cost reimbursable line item. 
7% Reduced the negotiation objective for profit or fee. 

12% Included a contract reopener clause that provides for adjustment of 
the contract amount after award. 

6% Other. 

Figure 8 shows that the majority of respondents stated they used additional cost 
analysis. The second most frequent response was allowing additional time to correct the 
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deficiency then resubmit a corrected proposal. Seventy-four percent of the responses 
identified only one action taken while 26% used two or more actions in tandem. 
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Figure 8. 

Analysis :This question was intended only for PCOs in the negotiations phase of a 
procurement. The responses to this question should help answer one of the subsidiary 
research questions of determining whether the alternatives provided in the DFARS motivate 
contractors so that the Government obtains a fair and reasonable contract. 

The use of additional cost analysis to determine the reasonableness of the cost 
element affected by the system’s deficiency was the most frequent response overall. Army 
and Navy PCOs used this option 35% and 29% of the time respectively. The Air Force 
ranked it as their second most frequent option. One Navy PCO provided a comment that this 
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option was difficult for some PCOs. This may be the case if time is of the essence. PCOs 
are dealing with a variety of timing issues on a regular basis that may preclude the use of this 
option. Expiring appropriations, political influences in the procurement process, and urgency 
of need are a few of the issues that may make this option unattractive. 

PCOs responding to the survey seem to use this option in tandem with others. Thirty- 
three percent of the time, when this option was chosen, another option was chosen with it. 
Every option offered in the survey was identified as an action used in tandem with additional 
cost analysis. The most frequent action used in concert with additional cost analysis was 
allowing the contractor additional time to correct the deficiency and resubmit a corrected 
proposal. 

Allowing the contractor additional time to correct the deficiency and submit a 
corrected proposal was the second most frequent response. The Army and the Navy used 
this option 25% and 21% respectively while the Air Force ranked it as their most frequent 
response with 33%. No particular command statistically stood out from the others. Analysis 
shows that of the 46 responses that used this option, 17 (37%) used it in combination with 
another action. This trend was seen equally among the three Services. When this option was 
used in combination with another, it was most frequently with the option of additional cost 
analysis. 

The option of “no change in plan” was included in the answer set for PCOs that were 
either dealing with estimating deficiencies that were minor and have no material affect on 
negotiations, or were dealing with a contractor who was a sole source. In these cases the 
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PCOs had no alternative but to continue with the process. The responses stating “no 
change” noted that either the deficiency was minor or the ACO was already working with 
the contractor to correct the system. One Navy PCO did note that he or she would only use 
this action when negotiating a cost type contract and the potential impact of fee is minimal. 

Including a contract reopener clause that provides for adjustment of the contract 
amount after award received 22 responses. Eight Navy, seven Army, four Air Force, and one 
DoD agency PCO used this option for the total of 22. The Navy and Army accounted for 1 8 
(82%) of the responses. Within the Navy, NAVSEA PCOs used this option six times. This 
appears to be a standard policy for NAVSEA PCOs. One NAVAIR PCO commented that 
this option has been used for years at NAVAIR; yet this was the only response from 
NAVAIR that identified this option. MICOM PCOs used this option six times. With one- 
third of the MICOM PCOs using this option it appears that this is somewhat of a standard 
procedure for this command. 

The researcher observed that PCOs use actions that would have the contractor fix the 
problem before an award is made rather than after. This can be seen in the fact that the use 
of additional time and additional cost analysis is used far more frequently than a reopener 
clause. A second reason for low overall response to this option is the fact that the DFARS 
requires that PCOs including a reopener clause are responsible for negotiating price 
adjustments required by the clause. This requires the PCO to be more involved in the 
administrative ftmctions of the contract than normal. PCOs prefer to award contracts after 
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contractors have fixed estimating problems rather than awarding and providing additional 
oversight for the deficient area. 

Reducing the negotiation objective for profit or fee received 13 (seven percent) 
responses. Six Navy, two Air Force, and one Army PCO used this option. All nine PCOs 
who responded used this option in concert with another option. There was no correlation 
between the commands using this option or the other option used. This option was used as 
part of the weighted guidelines under the cost control contractor risk factor. The evaluation 
criteria for below normal conditions are: 

1. The contractor’s cost estimating system is marginal. 

2. The contractor has made minimal effort to initiate cost reduction programs. 

3. The contractor’s cost proposal is inadequate. 

4. The contractor has a record of cost overruns or other indications of unreliable 

cost estimating and lack of cost control. 

PCOs using this option directly affect the companies’ profits. This is an option that assumes 
the contract will be awarded before the contractor fixes the estimating system problem. As 
seen by the relative low response, this confirms that PCOs prefer to have the estimating 
system fixed prior to award. 

Segregating the questionable areas as a cost-reimbursable line item has the second 
lowest response rate. It was used six times and four of them were as a combination of 
options. This option would be a good alternative if the contractor has an accounting system 
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deficiency. The cost-reimbursable feature allows the PCO visibility of all costs within that 
element thus keeping tabs on the contractor. 

Considering another contract type was the least used option. Three Air Force, one 
Navy, and one Army PCO identified this as a viable option. The low usage of this option 
could be due to the fact that other factors were more weighty in the selection of a contract 
type than the risk of inaccurately priced proposals. An example of this may be a research 
and development contract that has substantial development risk. Changing from a cost type 
contract to a firm fixed-priced type may be impracticable. 

Ten responses (five percent of the total) used other actions. This group was made up 
of five Navy, three Air Force, and two Army PCOs. They included a variety of comments 
and ideas such as: 

1 . The PCO stopped all activity with the contractor until the system was 
revalidated (three responses). 

2. Awarded the contract to the next competitor at a higher price (two responses). 

3. Documented the Business Clearance Memorandum (BCM) so the 
Government’s rights were protected in the event the deficiency results in a 

defective pricing action. 

4. Utilized a Department of Justice settlement of a fraud case to extract a 
commitment from the contractor to fix his deficiencies. 

5. Allowed ACO to include in a “global settlement” of issue (all active contracts 
and proposed effort). 
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6. Contractor withdrew proposal (two responses). 

Thirteen actions were taken by PCOs to reduce the profit objective and seven 
(included in the other categories) actions led to not awarding a contract to that company. 
These 20 (11%) actions taken by the PCO directly impacted the company’s profit and 
business base. This high percentage of PCOs who directly affect the businesses’ bottom line 
surprised the researcher. This severe action indicates that the contractors involved must have 
significant deficiencies. 

Examining the survey data as a whole shows that using additional cost analysis and 
allowing the contractor additional time to correct the deficiency then resubmit a corrected 
proposal were the most frequent responses by all Services. Together they account for 56% 
of the total responses received. The dimension of time plays a critical role in looking at these 
responses. Allowing the contractor additional time to correct the deficiency then resubmit 
a corrected proposal and using additional cost analysis adds to the procurement process time. 
Lengthening this process is counter to the current emphasis within DoD to reduce the time 
it takes to buy and field products. 

Question 8 & 9. Did the actions you took motivate the contractor to give the 
Government a fair and reasonable price? Why? 

a. Yes 

b. No 

Response: Survey results are: 

77% Yes 
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23% No 



A total of 43 respondents stated their actions did motivate the contractor and the 
Government did receive a fair and reasonable contract price. Fourteen respondents gave a 
negative answer. Additionally, 16 responses were received with this question unanswered. 
Figure 9 displays these results. 
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Figure 9. 

Analysis: These two questions were included in the questionnaire to evaluate if 
PCOs felt the actions they took in question 7 resulted in the Government receiving a fair and 
reasonable contract price and why. By analyzing the two questions together, six 
combinations of answers are possible. The researcher will analyze each combination 
individually. The first grouping of PCOs are ones that answered “yes” in question 8 and 
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gave comments in question 9. This group contained 35 responses that consisted of 1 8 Army, 
1 1 Navy, and six Air Force PCOs. Fifteen of these responses can be loosely grouped around 
the idea that the PCO believed the contractors were concerned about the loss of current or 
future business if the system’s problems were not fixed. Most of these comments stated that 
the PCOs were not going to award a contract until the deficiency was fixed or the 
disapproval was removed. These comments support the high usage of additional time to 
correct and additional cost analysis and low usage of reopener clauses, segregating the 
questionable area, and reducing profit or fee identified in question 7. This information 
indicates that the PCO is operating in an environment where time is not a critical element and 
the PCO can afford the time to allow the contractor to fix the estimating system. 

The second grouping consists of PCOs stating “yes” to question 8 but did not respond 
to question 9. This group was made up of four Army, two Navy, one Air Force, and one 
DoD agency PCO. Since no comments were provided by these responders the researcher can 
only assume that their comments would be similar to the previous group. 

The third group included PCOs that were in the negotiation phase but were unsure 
if their actions motivated the contractor. This group consisted of six Navy, four Army, and 
two Air Force PCOs. The majority of these responders stated that: (1) the deficiency was 
minor in impact, (2) they used additional cost analysis in question 7, or (3) they allowed the 
contractor additional time to correct the problem and resubmit the proposal. The researcher 
observed that PCOs taking a more neutral action such as two and three above were more 
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uncertain as to the affect their actions had on the contractor than PCOs that took a stronger 
stance as identified in the first group. 

The fourth group did not answer either question eight or nine because they were not 
in negotiations. This collection of answers consisted of two Air Force, one Army, and one 
Navy PCOs. 

The fifth group stated their actions did not motivate the contractor and did not 
provide a reason why. One response fell into this category. It was from an Army PCO who 
stated in question 7 that the deficiency did not have a material affect. The responder used 
the action “no change in plan” as the action taken in question 7. Since no action was taken, 
then the contractor was not motivated. 

The last collection of answers consisted of PCOs that did not feel their actions 
motivated the contractor and provided the reasons why they felt that way. This group 
included eight Navy, three Army, and two Air Force PCOs. The majority (11) from this 
group stated that the deficiency was not a factor in pricing or the contractor did not agree 
with DCAA. The two remaining PCOs were dealing with sole source contractors. These 
PCOs used more neutral actions of additional cost analysis and allowing the contractor 
additional time to correct the problem then resubmit the proposal. The researcher observed 
that when a PCO is dealing with a sole source they are more likely to use less aggressive 
actions toward the contractor than if operating in a multi-competitor market. 
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D. 



SUMMARY 



This chapter presented and analyzed the data obtained from the survey sent to various 
PCOs of large programs for the three Services and selected DoD agencies. This sample 
population was chosen because the researcher felt the members had the highest probability 
of dealing with contractor estimating systems. Overall, the comments received appeared to 
be open and informative. The level of responses was higher than the researcher anticipated. 

Analyzing the returned comments caused the researcher to rethink his goal of 
maintaining only a one page questionnaire. More in-depth information could have been 
obtained by including a few more questions despite any decline in the return rate. After 
evaluating the returned responses, the researcher found some weaknesses with the 
questionnaire. First, it did not give PCOs that had not ever been “officially” notified of a 
deficient contractor estimating system the ability to provide information on actions they may 
have taken to protect the Government anyway. Secondly, the questionnaire did not ask the 
PCO to provide their rationale on why they took the actions they did. Lastly, the 
questionnaire failed to direct the respondents to provide only one answer for each occurrence 
in question 5. 

Chapter V will provide conclusions, recommendations, answers to the research 
questions, and recommendations for further research. 
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V. CONCLUSIONS AND RECOMMENDATIONS 



A. INTRODUCTION 

Reliable cost estimating systems are a key safeguard for the Government to obtain 
fair and reasonable contract prices. When a contract price is higher because it is based on 
inaccurate, incomplete, and non-current data the Government has the right to a price 
reduction under the provisions of TINA. When contracts are overpriced because of poor 
estimating the Government may not recover the overstated price under TINA. Numerous 
audits conducted by GAO over the last several years indicate there are significant problems 
and weaknesses in many contractors’ cost estimating systems. 

The objectives of this resecurch effort were: (1) to determine how PCOs obtain 
information of a contractor poor cost estimating system and (2) to determine how this 
information is used by the PCO once obtained. For the Government to receive a fair and 
reasonable contract price the PCO must know if the contractor’s proposal was generated by 
an adequate estimating system. If the PCO is unaware of the adequacy of the system the 
Government cannot recover any overstated amounts from the contractor once the negotiation 
is completed. The information about the adequacy of a contractor’s estimating system is a 
key defense that PCOs must use to ensure taxpayers funds are spent wisely. The researcher 
conducted a survey of PCOs in high dollar value DoD programs and projects to achieve the 
stated objectives. These PCOs were sent a nine-question survey estimated to take ten 
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minutes to complete. The researcher felt that a brief questionnaire would increase the 



response rate. 

B. CONCLUSIONS 

This section discusses conclusions drawn as a result of this research effort. 

1. PCOs are informed of deficient contractor estimating systems by the ACO 
and DCAA auditor. 

The procedures in the DEARS and DLA directives place the responsibility of 
estimating system adequacy with the ACO. The DCAA auditors are to assist the ACO with 
this function by conducting estimating system surveys once every three years. These 
procedures were discussed in earlier chapters of this thesis. The results of the researcher’s 
survey indicate that the ACO and DCAA auditor inform PCOs the vast majority of the time. 
For the research sample, this accounted for 91%. This confirms that the DFARS and DLA 
procedures discussed in Chapter II are being followed. 

2. PCOs are typically notified of poor estimating systems during proposal 
evaluation and negotiation situations. 

The DFARS requires PCOs to request field pricing reports for procurement actions 
over prescribed thresholds. Many of the PCOs stated they had been informed of the 
adequacy of the contractor’s estimating system via these reports. Additionally, some ACOs 
include their semi-annual ACO memorandum updating the status of the contractor’s 
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corrective action plan in the report. For the sample population, this method was used 51% 
of the time. 

3. PCOs take actions internal to the Government when notified of estimating 
systems problems. 

PCOs typically take actions internal to the Government when notified of contractors’ 
deficient estimating systems. These actions include determining the impact of the deficiency 
or disapproval, holding discussions with the ACO and DCAA auditor, and informing the 
PCO’s chain of command. The impact of the deficient system is based on the contractual 
relationship between the PCO and the contractor. Discussions with the ACO and DCAA are 
generally conducted to determine the status of the corrective actions performed by the 
contractor. The PCOs informed their chain of command in an attempt to formulate a 
coordinated response from a command perspective. The survey conducted for this research 
determined that 62% of the actions taken by the survey participates were internal to the 
Government. 

4. Generally PCOs use either additional cost analysis or additional time to 
correct systems deficiencies when in negotiations. 

The DFARS provides various alternative action for PCO to use when dealing with 
a contractor that has a deficient estimating system. PCOs in the negotiations phase of a 
procurement, utilize additional cost analysis on proposals generated from these deficient 
estimating systems. Secondly, they allow the contractor additional time to correct the 
deficiency and submit a corrected proposal. Fifty-six percent of the sample participants used 
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one of these two methods. Both actions may lengthen the negotiation phase of a 
procurement. Utilizing additional cost analysis requires additional resources to be used when 
they could be focused in other areas. 

5. PCOs believe their actions motivated the contractor so that the Government 
received a fair and reasonable price. 

PCOs use a wide range of actions to ensure contractors are sensitive to the 
Government’ s concerns. One of these concerns is fair and reasonable contract prices. When 
the contract price is negotiated based on a deficient estimate, the PCO must take actions to 
protect the Government. The majority (77%) of PCOs in the sample believe that the actions 
they took motivated the contractor so the Government obtained a fair and reasonable price. 
Although the rationale given varied greatly, many PCOs felt contractors were concerned 
about losing current or future business if corrections to their estimating systems were not 
made. 

6. PCOs do not receive notification about contractor estimating system 
problems from their contracting chain of command. 

This conclusion is based on three pieces of independent data provided in the returned 
surveys. First, none of the PCOs listed their contracting chain of command (buying 
command management or contracting directorate) as a source of information about a 
contractor’s estimating system in question 5. Secondly, four separate PCOs stated that they 
had informed their upper management so that a coordinated command or Service response 
could be formulated in question 6. Finally, ten of the 40 commands returning surveys had 
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some PCOs that were notified of estimating system problems and some not. Examining each 
bit of information independently does not allow for a specific conclusion to be formed. 
However, when analyzed together, they indicate that the Government could be taking a more 
coordinated effort to ensure a fair and reasonable contract price is reached. 

C. RECOMMENDATIONS 

This section discusses the recommendations concluded from this research effort. 

1. DCAA should provide all buying commands with an annual listing of 
contractors evaluated as high risk, based on DCAA’s Internal Control Audit Planning 
Summary (ICAPS) System. 

Providing this kind of information would offer PCOs more data upon which to 
determine if additional cost analysis should be conducted. Using these data, PCOs could 
evaluate the resources available and time limits of each procurement against the risk of an 
inaccurate contractorproposal. Resources would be more focused on the proposals that truly 
represent a risk to the Government. 

2. Buying commands management and contracting directorates should act as 
a conduit for information regarding contractors estimating systems. 

PCOs have identified deficient contractor systems to their chain of commands but 
none of the PCOs identified their chain of command as a source of information about these 
systems. Encouraging the management structure to act as a conduit and coordinate a more 
consolidated and synergistic response toward a specific contractor could be achieved. The 
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contractor would have more impetus to correct the deficiencies or seek approval of his 
estimating system. By reducing the number of contractors with problem systems, savings 
can be achieved by decreasing ACO and DCAA workloads. 

3. PCOs should be encouraged not to award to firms with deficient or 
disapproved estimating systems. 

As seen from the research, PCOs use actions that lengthen the contract process and 
expend additional resources when dealing with contractors that have problem systems. The 
Army Contracting Officer example in Chapter III is the perfect case. He submitted 
approximately 1 00 requests to the contractor for additional information. This was more than 
twice the normal number. It takes valuable resources from both the Government and the 
contractor to generate, answer and evaluate these requests. By not awarding to these 
contractors, the Government saves time, saves resources, and sends a strong message to 
contractors. 

D. SUMMARY OF RESEARCH QUESTIONS 

In order to accomplish the objectives of this study, the following research questions 
were pursued; 

1. Primary research question. What actions does the Procuring Contracting 
Officer take when notified that a contractor’s estimating system has been disapproved 
by the Administrative Contracting Officer or has deficiencies? 
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In general, PCOs take actions internal to the Government when notified of contracted 
estimating systems being disapproved or having deficiencies. These actions include 
determining the impact of the disapproval or deficiency, holding discussions with the AGO 
and DC AA auditor, and informing their management structure. In the surveyed population, 
PCOs took actions internal to the Government 62% of the time. When specifically in the 
negotiation phase of a procurement, PCOs use a combination of additional cost analysis and 
allowing the contractor additional time to correct the problem and resubmit a corrected 
proposal. These actions were taken 56% of the time for the sample population. 

2. Subsidiary research question #1. Are the alternative actions specified in 
DFARS 215.81 l-70(g)(2) adequate to motivate a contractor to respond to the PCO’s 
desires? 

Yes, the alternative actions in the DFARS are adequate to motivate contractors. The 
survey shows that the majority of PCOs believed that the contractor was motivated by the 
action they took. Many PCOs expressed contractors are concerned about the impact their 
deficient estimating system has on current and future business therefore they respond to the 
PCO’s actions. 

3. Subsidiary research question #2. Are there more effective actions currently 
being used by PCOs and if so should they be incorporated into the DFARS? 

There is one action currently being used by PCOs that is not listed in the DFARS. 
This action is to award the contract to the next competitor at a presumably higher contract 
price. It cannot be determined if this action is more effective than current DFARS actions 



69 



but the responding PCOs stated they believe it is effective. In the opinion of the researcher, 
this option is a good one but should not be included in the DFARS. By putting this option 
in the DFARS some PCOs will take it as a required action. That is not the intent. The intent 
is that it should be used if the situation warrants. 

4. Subsidiary research question #3. If the PCO gathers independent data 
indicating a contractor has estimating system deficiencies, how does this affect the 
performance of his or her duties? 

Only two PCOs identified they had gathered independent data about contractors’ 
estimating systems. Insufficient data were provided to be able to evaluate the affects on 
PCOs actions. It is unclear whether this was caused by a fault in the questionnaire or few 
PCOs gather independent data about contractor estimating systems. 

E. ADDITIONAL AREAS OF RESEARCH 

As a result of the research conducted on contractors’ estimating systems, the 
following areas warrant further investigation: 

1 . Survey buying command management structitres regarding information flow in 
regards to deficient estimating systems within the command. 

2. Survey Administrative Contacting Offices regarding affect of PCO actions on a 
contractor. 

3. Analyze DCMC headquarters current oversight of regional offices in terms of 
reporting deficiencies and disapprovals and adequacy of follow up reporting systems. 
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4. Survey industry regarding whether the current estimating system regulations and 
oversight structure are too burdensome and if industry can suggest alternatives. 

5. Analysis ACO adequacy determination decisions of contractor estimating systems. 
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APPENDIX A. PCO SURVEY 



Directions: Please complete the following information. 

1 . How many years of contracting experience do you have? 

a. [ ]0-3 b. [ ]4-8 c. [ ]9-15 d. [ ] 16-25 e. [ ] 26 or greater 

2. What is the average dollar value of contracts you are responsible for per year? 

a. [ ]0-$ 1,000, 000 b. [ ] $1,000,000-$ 10,000,000 

c. [ ]$10,000,000-$50,000,000 d. [ ] $50,000,000-$ 100,000,000 

e. [ ] $100,000,000 or greater 

3. Have you ever received notification of a contractor’s estimating system having 
deficiencies or being disapproved? 

a. [ ] yes b. [ ] no 

4. Why were you notified? 



If yes, please discuss the last three occurrences: 



5. How did you learn of the 
deficiencies or disapproval? 


#1 


#2 


#3 


AGO 


[ ] 


[ ] 


[ ] 


DCAA 


[ ] 


[ ] 


[ ] 


Contractor 


[ ] 


[ ] 


[ ] 


Cost Analyst 


[ ] 


[ ] 


[ ] 


Govt. Tech/Eng 
Other (specify) 


[ ] 


[ ] 


[ ] 



6. What action did you take once notified? 

#1 

#2 

#3 
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7. If in negotiations, how did you proceed? 

a. No change in plan. 

b. Allowed the contractor additional time to correct the 
deficiency and submit a corrected proposal. 

c. Considered another type of contract. 

d. Used additional cost analysis to determine the 
reasonableness of the cost elements affected by the system’s 
deficiency. 

e. Segregated the questionable areas as a cost reimbursable 
line item. 

f. Reduced the negotiation objective for profit or fee. 

g. Included a contract reopener clause that provides for 
adjustment of the contract amount after award. 



#1 


#2 


#3 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 


[ ] 



h. Other (specify) 
#1 



#2 



#3 



8. Did the actions you took motivate the contractor to give the Government a fair and 
reasonable price? 

a. [ ] yes b. [ ] no 



9. Why? 



Optional: If you have any questions about the survey or want to discuss any of your 
answers please print your name and telephone or E-mail address and I will contact you. 

Name: 

Phone number or E-mail address: 

Thank you for your participation! 



78 



APPENDIX B. SURVEY RESPONSES 



The following tables are the aggregate responses received from the researchers survey. 
Question 1 . How many years of contracting experience do you have? 



Command 


la 


lb 


Ic 


Id 


le 


Advanced Research Projects Agency 


0 


0 


0 


1 


0 


Aeronautical Systems Center 


0 


0 


0 


2 


1 


Air Force Development Test Center 


0 


0 


0 


1 


0 


Air Force Material Command 


0 


0 


1 


1 


0 


Air Force Office of Scientific Research 


0 


0 


0 


1 


0 


Airborne Warning & Control Systems 


0 


0 


0 


1 


0 


Area Attack SPO 


0 


0 


0 


1 


0 


Armament Research, Development & Engineering Center 


0 


0 


0 


3 


0 


Arnold Engineering Development Center 


0 


0 


0 


1 


0 


Aviation & Troop Command 


0 


0 


2 


3 


1 


B-2 Program Office 


0 


0 


0 


1 


0 


Chemical & Biological Defense Command 


0 


0 


0 


0 


1 


Cheyenne Mountain Complex System 


0 


0 


0 


0 


1 


Communications Electronics Command 


0 


0 


0 


4 


0 


DOD Ballistic Missile Defense Organization 


0 


0 


0 


1 


0 


Electronic Systems Center 


0 


0 


1 


2 


0 


F-16 SPO 


0 


0 


0 


0 


1 


F-22 System Program Office 


0 


0 


0 


0 


1 


Joint Stars 


0 


0 


0 


1 


0 


Joint Tactical Information Distribution System 


0 


0 


0 


1 


0 


Marine Corps Systems Command 


0 


0 


0 


1 


0 


Missile Command 


0 


1 


1 


9 


3 
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Command 


la 


lb 


Ic 


Id 


le 


Naval Air Systems Command 


1 


0 


5 


6 


1 


Naval Research Laboratory 


0 


0 


0 


1 


0 


Naval Sea Systems Command 


0 


1 


4 


8 


2 


Naval Surface Warfare Center 


0 


0 


0 


1 


1 


Naval Undersea Warfare Center 


0 


0 


0 


0 


1 


Navy Inventory Control Point 


0 


0 


0 


0 


1 


Office of Naval Research 


0 


0 


2 


0 


1 


Ogden Air Logistics Center 


0 


0 


0 


0 


1 


Phillips Laboratory 


0 


1 


0 


1 


0 


San Antonio Air Logistics Center 


1 


0 


1 


0 


0 


Space & Missile Systems Center 


0 


0 


1 


3 


0 


Space and Naval Warfare Systems Command 


0 


0 


0 


2 


1 


Space & Strategic Defense Command 


0 


0 


0 


1 


0 


Strategic Systems Programs 


0 


0 


0 


1 


0 


US Army Space Command 


0 


0 


0 


0 


1 


Utility Helicopters 


0 


0 


1 


0 


0 


Warner Robins Air Logistics Center 


0 


0 


1 


2 


0 


Wright Laboratories 


0 


0 


1 


0 


0 
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Question 2. What is the average dollar value of contracts you are responsible for per 
year? 



Command 


2a 


2b 


2c 


2d 


2e 


Advanced Research Projects Agency 


0 


0 


1 


0 


0 


Aeronautical Systems Center 


0 


0 


0 


1 


2 


Air Force Development Test Center 


0 


0 


0 


0 


1 


Air Force Material Command 


0 


1 


0 


1 


0 


Air Force Office of Scientific Research 


1 


0 


0 


0 


0 


Airborne Warning & Control Systems 


0 


0 


0 


0 


1 


Area Attack SPO 


0 


0 


1 


0 


0 


Armament Research, Development & Engineering Center 


0 


0 


1 


1 


1 


Arnold Engineering Development Center 


0 


0 


0 


0 


1 


Aviation & Troop Command 


0 


0 


1 


3 


2 


B-2 Program Office 


0 


0 


0 


0 


1 


Chemical & Biological Defense Command 


0 


1 


0 


0 


0 


Cheyenne Mountain Complex System 


0 


0 


0 


0 


1 


Communications Electronics Command 


0 


0 


0 


2 


2 


DOD Ballistic Missile Defense Organization 


0 


0 


1 


0 


0 


Electronic Systems Center 


0 


0 


2 


0 


1 


F-16 SPO 


0 


0 


0 


1 


0 


F-22 System Program Office 


0 


0 


0 


0 


1 


Joint Stars 


0 


0 


0 


0 


1 


Joint Tactical Information Distribution System 


0 


0 


1 


0 


0 


Marine Corps Systems Command 


0 


0 


0 


0 


1 


Missile Command 


2 


2 


2 


2 


6 


Naval Air Systems Command 


0 


0 


2 


4 


7 
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Command 


2a 


2b 


2c 


2d 


2e 


Naval Research Laboratory 


0 


0 


0 


0 


1 


Naval Sea Systems Command 


0 


2 


2 


1 


10 


Naval Surface Warfare Center 


0 


0 


1 


0 


1 


Naval Undersea Warfare Center 


0 


1 


0 


0 


0 


Navy Inventory Control Point 


1 


0 


0 


0 


0 


Office of Naval Research 


0 


1 


0 


0 


2 


Ogden Air Logistics Center 


1 


0 


0 


0 


0 


Phillips Laboratory 


0 


0 


1 


1 


0 


San Antonio Air Logistics Center 


1 


1 


0 


0 


0 


Space & Missile Systems Center 


0 


0 


0 


0 


4 


Space and Naval Warfare Systems Command 


0 


0 


0 


1 


2 


Space & Strategic Defense Command 


0 


0 


1 


0 


0 


Strategic Systems Programs 


0 


0 


1 


0 


0 


US Army Space Command 


0 


0 


1 


0 


0 


Utility Helicopters 


0 


0 


0 


0 


1 


Warner Robins Air Logistics Center 


1 


1 


0 


0 


1 


Wright Laboratories 


0 


0 


1 


0 


0 
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Question 3. Have you ever received notification of a contractors estimating system 
having deficiencies or being disapproved? 



Command 


3a 


3b 


Advanced Research Projects Agency 


0 


1 


Aeronautical Systems Center 


2 


1 


Air Force Development Test Center 


0 


1 


Air Force Material Command 


2 


0 


Air Force Office of Scientific Research 


1 


0 


Airborne Warning & Control Systems 


1 


0 


Area Attack SPO 


1 


0 


Armament Research, Development & Engineering Center 


1 


2 


Arnold Engineering Development Center 


0 


1 


Aviation & Troop Command 


4 


2 


B-2 Program Office 


0 


1 


Chemical & Biological Defense Command 


1 


0 


Cheyenne Mountain Complex System 


1 


0 


Communications Electronics Command 


4 


0 


DOD Ballistic Missile Defense Organization 


1 


0 


Electronic Systems Center 


1 


2 


F-16SPO 


1 


0 


F-22 System Program Office 


1 


0 


Joint Stars 


1 


0 


Joint Tactical Information Distribution System 


1 


0 


Marine Corps Systems Command 


1 


0 


Missile Command 


12 


2 


Naval Air Systems Command 


8 


5 
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Command 


3a 


3b 


Naval Research Laboratory 


1 


0 


Naval Sea Systems Command 


12 


3 


Naval Surface Warfare Center 


2 


0 


Naval Undersea Warfare Center 


1 


0 


Navy Inventory Control Point 


1 


0 


Office of Naval Research 


0 


3 


Ogden Air Logistics Center 


1 


0 


Phillips Laboratory 


1 


1 


San Antonio Air Logistics Center 


1 


1 


Space & Missile Systems Center 


1 


3 


Space and Naval Warfare Systems Command 


1 


2 


Space & Strategic Defense Command 


1 


0 


Strategic Systems Programs 


1 


0 


US Army Space Command 


0 


1 


Utility Helicopters 


0 


1 


Warner Robins Air Logistics Center 


3 


0 


Wright Laboratories 


1 


0 
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Question 4. Why were you notified? 



Command 


1 


2 


3 


4 


Advanced Research Projects Agency 


0 


0 


0 


0 


Aeronautical Systems Center 


0 


1 


1 


0 


Air Force Development Test Center 


0 


0 


0 


0 


Air Force Material Command 


1 


0 


0 


1 


Air Force Office of Scientific Research 


1 


0 


0 


0 


Airborne Warning & Control Systems 


0 


1 


0 


0 


Area Attack SPO 


0 


0 


0 


1 


Armament Research, Development & Engineering Center 


1 


0 


0 


0 


Arnold Engineering Development Center 


0 


0 


0 


0 


Aviation & Troop Command 


0 


4 


0 


0 


B-2 Program Office 


0 


0 


0 


0 


Chemical & Biological Defense Command 


1 


0 


0 


0 


Cheyenne Mountain Complex System 


1 


0 


0 


0 


Communications Electronics Command 


3 


1 


0 


0 


DOD Ballistic Missile Defense Organization 


1 


0 


0 


0 


Electronic Systems Center 


0 


1 


0 


0 


F- 16 SPO 


0 


1 


0 


0 


F-22 System Program Office 


1 


0 


0 


0 


Joint Stars 


0 


1 


0 


0 


Joint Tactical Information Distribution System 


1 


0 


0 


0 


Marine Corps Systems Command 


0 


1 


0 


0 


Missile Command 


8 


4 


0 


0 


Naval Air Systems Command 


2 


2 


2 


2 


Naval Research Laboratory 


0 


1 


0 


0 
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Command 


1 


2 


3 


4 


Naval Sea Systems Command 


6 


6 


'o 


0 


Naval Surface Warfare Center 


2 


0 


0 


0 


Naval Undersea Warfare Center 


1 


0 


0 


0 


Navy Inventory Control Point 


0 


0 


1 


0 


Office of Naval Research 


0 


0 


0 


0 


Ogden Air Logistics Center 


0 


1 


0 


0 


Phillips Laboratory 


1 


0 


0 


0 


San Antonio Air Logistics Center 


1 


0 


0 


0 


Space & Missile Systems Center 


0 


1 


0 


0 


Space and Naval Warfare Systems Command 


1 


0 


0 


0 


Space & Strategic Defense Command 


0 


0 


0 


1 


Strategic Systems Programs 


1 


0 


0 


0 


US Army Space Command 


0 


0 


0 


0 


Utility Helicopters 


0 


0 


0 


0 


Warner Robins Air Logistics Center 


2 


1 


0 


0 


Wright Laboratories 


1 


0 


0 


0 



Note: 

1 . Result of a proposal evaluation request. 

2. PCO on standard distribution list or PCO of record. 

3. Miscellaneous. 

4. No response 
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Question 5. How did you leam of the deficiencies or disapproval? 



Command 


5a 


5b 


5c 


5d 


5e 


5f 


Advanced Research Projects Agency 


0 


0 


0 


0 


0 


0 


Aeronautical Systems Center 


4 


0 


0 


0 


0 


0 


Air Force Development Test Center 


0 


0 


0 


0 


0 


0 


Air Force Material Command 


2 


1 


0 


1 


0 


0 


Air Force Office of Scientific Research 


1 


1 


0 


0 


0 


0 


Airborne Warning & Control Systems 


1 


1 


0 


0 


0 


0 


Area Attack SPO 


1 


1 


0 


0 


0 


0 


Armament Research, Development & Engineering Center 


1 


1 


0 


0 


0 


0 


Arnold Engineering Development Center 


0 


0 


0 


0 


0 


0 


Aviation & Troop Command 


3 


2 


0 


0 


1 


0 


B-2 Program Office 


0 


0 


0 


0 


0 


0 


Chemical & Biological Defense Command 


0 


3 


0 


0 


0 


0 


Cheyenne Mountain Complex System 


0 


3 


0 


0 


0 


0 


Communications Electronics Command 


4 


6 


0 


0 


0 


0 


DOD Ballistic Missile Defense Organization 


0 


1 


0 


0 


0 


0 


Electronic Systems Center 


1 


0 


0 


0 


0 


0 


F-16SPO 


2 


0 


0 


0 


0 


0 


F-22 System Program Office 


1 


2 


0 


0 


0 


0 


Joint Stars 


1 


0 


0 


0 


0 


0 


Joint Tactical Information Distribution System 


3 


3 


0 


0 


0 


0 


Marine Corps Systems Command 


1 


0 


0 


0 


0 


1 


Missile Command 


6 


21 


3 


0 


0 


0 


Naval Air Systems Command 


10 


6 


1 


1 


0 


0 


Naval Research Laboratory 


0 


1 


0 


0 


0 


0 
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Command 


5a 


5b 


5c 


5d 


5e 


5f 


Naval Sea Systems Command 


17 


15 


1 


0 


0 


0 


Naval Surface Warfare Center 


3 


3 


1 


0 


0 


0 


Naval Undersea Warfare Center 


0 


1 


0 


0 


0 


0 


Navy Inventory Control Point 


1 


0 


1 


0 


0 


0 


Office of Naval Research 


0 


0 


0 


0 


0 


0 


Ogden Air Logistics Center 


0 


3 


0 


0 


0 


0 


Phillips Laboratory 


1 


1 


0 


0 


0 


0 


San Antonio Air Logistics Center 


0 


3 


0 


0 


0 


0 


Space & Missile Systems Center 


0 


0 


0 


0 


0 


0 


Space and Naval Warfare Systems Command 


1 


1 


0 


0 


0 


0 


Space & Strategic Defense Command 


0 


1 


0 


0 


0 


0 


Strategic Systems Programs 


0 


3 


0 


0 


0 


0 


US Army Space Command 


0 


0 


0 


0 


0 


0 


Utility Helicopters 


0 


0 


0 


0 


0 


0 


Warner Robins Air Logistics Center 


7 


3 


3 


3 


0 


0 


Wright Laboratories 


2 


1 


0 


0 


0 


0 
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Question 6. What actions did you take once notified? 



Command 


1 


2 


3 


4 


Advanced Research Projects Agency 


0 


0 


0 


0 


Aeronautical Systems Center 


0 


3 


3 


0 


Air Force Development Test Center 


0 


0 


0 


0 


Air Force Material Command 


6 


0 


0 


0 


Air Force Office of Scientific Research 


1 


0 


1 


0 


Airborne Warning & Control Systems 


0 


2 


0 


0 


Area Attack SPO 


1 


0 


0 


0 


Armament Research, Development & Engineering Center 


0 


2 


0 


0 


Arnold Engineering Development Center 


0 


0 


0 


0 


Aviation & Troop Command 


3 


3 


0 


0 


B-2 Program Office 


0 


0 


0 


0 


Chemical & Biological Defense Command 


2 


0 


1 


0 


Cheyenne Mountain Complex System 


3 


0 


0 


0 


Communications Electronics Command 


1 


2 


5 


0 


DOD Ballistic Missile Defense Organization 


1 


0 


0 


0 


Electronic Systems Center 


0 


0 


1 


0 


F-16 SPO 


0 


2 


0 


0 


F-22 System Program Office 


3 


0 


0 


0 


Joint Stars 


0 


0 


2 


0 


Joint Tactical Information Distribution System 


3 


0 


0 


0 


Marine Corps Systems Command 


3 


0 


0 


0 


Missile Command 


22 


1 


3 


2 


Naval Air Systems Command 


8 


2 


6 


2 


Naval Research Laboratory 


1 


0 


0 


0 
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Command 


1 


2 


3 


4 


Naval Sea Systems Command 


12 


5 


10 


0 


Naval Surface Warfare Center 


2 


2 


0 


1 


Naval Undersea Warfare Center 


1 


0 


0 


0 


Navy Inventory Control Point 


1 


0 


0 


0 


Office of Naval Research 


0 


0 


0 


0 


Ogden Air Logistics Center 


3 


0 


0 


0 


Phillips Laboratory 


2 


0 


0 


0 


San Antonio Air Logistics Center 


2 


1 


0 


0 


Space & Missile Systems Center 


2 


0 


0 


0 


Space and Naval Warfare Systems Command 


0 


0 


2 


0 


Space & Strategic Defense Command 


0 


1 


0 


0 


Strategic Systems Programs 


3 


0 


0 


0 


US Army Space Command 


0 


0 


0 


0 


Utility Helicopters 


0 


0 


0 


0 


Warner Robins Air Logistics Center 


4 


0 


0 


1 


Wright Laboratories 


3 


0 


0 


0 



Note: 

1 . Action internal to the Government. 

2. Action directly with the contractor. 

3. Action including the contractor and the Government. 

4. Miscellaneous. 
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Question 7. If in negotiations how did you proceed? 



Command 


7a 


7b 


7c 


7d 


7e 


7f 


7g 


7h 


Advanced Research Projects Agency 


0 


0 


0 


0 


0 


0 


0 


0 


Aeronautical Systems Center 


1 


1 


0 


1 


1 


0 


1 


0 


Air Force Development Test Center 


0 


0 


0 


0 


0 


0 


0 


0 


Air Force Material Command 


0 


2 


0 


2 


0 


0 


0 


0 


Air Force Office of Scientific Research 


0 


1 


0 


1 


0 


0 


0 


0 


Airborne Warning & Control Systems 


0 


1 


0 


0 


0 


0 


0 


0 


Area Attack SPO 


0 


0 


0 


1 


0 


0 


0 


0 


Armament Research, Development & 
Engineering Center 


0 


1 


0 


1 


0 


1 


0 


0 


Arnold Engineering Development Center 


0 


0 


0 


0 


0 


0 


0 


0 


Aviation & Troop Command 


0 


2 


0 


4 


0 


0 


1 


0 


B-2 Program Office 


0 


0 


0 


0 


0 


0 


0 


0 


Chemical & Biological Defense Command 


0 


1 


0 


2 


0 


0 


0 


0 


Cheyenne Mountain Complex System 


3 


0 


0 


0 


0 


0 


0 


0 


Communications Electronics Command 


0 


5 


1 


2 


1 


0 


2 


0 


DOD Ballistic Missile Defense Organization 


0 


0 


0 


1 


0 


0 


1 


0 


Electronic Systems Center 


0 


0 


0 


0 


0 


0 


0 


0 


F-16SPO 


0 


2 


0 


0 


0 


0 


0 


0 


F-22 System Program Office 


0 


1 


0 


1 


0 


0 


0 


1 


Joint Stars 


0 


1 


1 


1 


0 


0 


0 


0 


Joint Tactical Information Distribution System 


1 


0 


0 


0 


0 


0 


0 


0 


Marine Corps Systems Command 


0 


0 


0 


1 


0 


0 


0 


0 


Missile Command 


11 


5 


0 


11 


0 


0 


6 


1 


Naval Air Systems Command 


2 


3 


0 


6 


0 


6 


1 


4 


Naval Research Laboratory 


0 


3 


0 


0 


0 


0 


0 


0 
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Command 


7a 


7b 


7c 


7d 


7e 


7f 


7g 


7h 


Naval Sea Systems Command 


4 


3 


0 


7 


0 


1 


6 


2 


Naval Surface Warfare Center 


0 


3 


1 


3 


2 


2 


0 


0 


Naval Undersea Warfare Center 


1 


0 


0 


1 


0 


1 


0 


0 


Navy Inventory Control Point 


0 


1 


0 


1 


0 


0 


1 


0 


Office of Naval Research 


0 


0 


0 


0 


0 


0 


0 


0 


Ogden Air Logistics Center 


0 


0 


1 


0 


1 


0 


1 


0 


Phillips Laboratory 


0 


0 


1 


0 


0 


1 


1 


0 


San Antonio Air Logistics Center 


0 


3 


0 


3 


0 


0 


0 


0 


Space & Missile Systems Center 


0 


0 


0 


0 


0 


0 


0 


2 


Space and Naval Warfare Systems Command 


0 


2 


0 


0 


0 


0 


0 


0 


Space & Strategic Defense Command 


0 


1 


0 


1 


0 


0 


0 


0 


Strategic Systems Programs 


0 


0 


0 


2 


0 


0 


1 


0 


US Army Space Command 


0 


0 


0 


0 


0 


0 


0 


0 


Utility Helicopters 


0 


0 


0 


0 


0 


0 


0 


0 


Warner Robins Air Logistics Center 


0 


4 


0 


1 


0 


1 


0 


0 


Wright Laboratories 


1 


0 


0 


1 


1 


0 


0 


0 
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Question 8. Did the actions you took motivate the contractor to give the Government a 
fair and reasonable price? 



Command 


yes 


no 


Advanced Research Projects Agency 


0 


0 


Aeronautical Systems Center 


2 


0 


Air Force Development Test Center 


0 


0 


Air Force Material Command 


1 


0 


Air Force Office of Scientific Research 


1 


0 


Airborne Warning & Control Systems 


1 


0 


Area Attack SPO 


0 


0 


Armament Research, Development & Engineering Center 


1 


0 


Arnold Engineering Development Center 


0 


0 


Aviation & Troop Command 


3 


1 


B-2 Program Office 


0 


0 


Chemical & Biological Defense Command 


0 


1 


Cheyenne Mountain Complex System 


1 


0 


Communications Electronics Command 


2 


0 


DOD Ballistic Missile Defense Organization 


1 


0 


Electronic Systems Center 


0 


0 


F-16 SPO 


1 


0 


F-22 System Program Office 


1 


0 


Joint Stars 


1 


0 


Joint Tactical Information Distribution System 


1 


0 


Marine Corps Systems Command 


0 


0 


Missile Command 


7 


2 


Naval Air Systems Command 


4 


3 
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Command 


yes 


no 


Naval Research Laboratory 


1 


0 


Naval Sea Systems Command 


4 


4 


Naval Surface Warfare Center 


2 


0 


Naval Undersea Warfare Center 


1 


0 


Navy Inventory Control Point 


0 


1 


Office of Naval Research 


0 


0 


Ogden Air Logistics Center 


0 


1 


Phillips Laboratory 


0 


1 


San Antonio Air Logistics Center 


1 


0 


Space & Missile Systems Center 


1 


0 


Space and Naval Warfare Systems Command 


0 


0 


Space & Strategic Defense Command 


1 


0 


Strategic Systems Programs 


1 


0 


US Army Space Command 


0 


0 


Utility Helicopters 


0 


0 


Warner Robins Air Logistics Center 


2 


0 


Wright Laboratories 


1 


0 
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